Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “integral experiment”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,423 records · Page 79
All sourcesNASA NTRSDOE OSTI
DOE OSTI2026

iDDS: intelligent distributed dispatch and scheduling for workflow orchestration

The intelligent distributed dispatch and scheduling (iDDS) service is a versatile workflow orchestration system designed for large-scale, distributed scientific computing. iDDS extends traditional workload and data management by integrating data-aware execution, conditional logic, and programmable workflows, enabling automation of complex and dynamic processing pipelines. Originally developed for the ATLAS experiment at the large hadron collider, iDDS has evolved into an experiment-agnostic platform that supports both template-driven workflows and a Function-as-a-Task model for Python-based orchestration. This paper presents the architecture and core components of iDDS, highlighting its scalability, modular message-driven design, and integration with systems such as PanDA and Rucio. We demonstrate its versatility through real-world use cases: fine-grained tape resource optimization for ATLAS, orchestration of large Directed Acyclic Graph (DAG) workflows for the Rubin Observatory, distributed hyperparameter optimization for machine learning applications, active learning for physics analyses, and AI-assisted detector design at the electron–ion collider. By unifying workload scheduling, data movement, and adaptive decision-making, iDDS reduces operational overhead and enables reproducible, high-throughput workflows across heterogeneous infrastructures. We conclude with current challenges and future directions, including interactive, cloud-native, and serverless workflow support.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

BISON validation to in situ cladding burst test and high-burnup LOCA experiments

The process for developing and qualifying nuclear fuels for commercial nuclear application requires fundamental material development, characterization, and design; out-of-pile testing on unirradiated materials; integral fuel rod irradiations, testing, and postirradiation examinations; and transient analyses. The historical approach depends on the generation of large empirical datasets and series of integral fuel rod irradiations, and this approach ultimately takes ~20 years—or sometimes longer—to acquire data through extensive sequential testing. Thus, the qualification and eventual deployment of new fuel systems constitute a long process. However, recent technological advancements have provided researchers the opportunity to perform out-of-cell, in situ measurements to assess material performance for the duration of the experiment. One such example of this capability is the use of digital image coordination and thermal imaging to assess Zircaloy cladding performance under a simulated loss-of-coolant accident (LOCA) transient condition. In situ measurements generally provide high-fidelity strain, strain rates, and temperature surface maps. This is critical for the US nuclear industry, which is actively developing a technical basis to support extending the peak rod average burnup from 62 to ~75 GWd/tU and the deployment of accident-tolerant fuel. However, the US Nuclear Regulatory Commission (NRC) outlined in its research information letter several technical issues that the industry must address before extending burnup. One topic of specific interest is understanding the cladding balloon and rupture geometry during the LOCA heat-up phase. By leveraging these advanced in situ capabilities, this work used in situ data generated from a simulated LOCA to better understand high-temperature creep and its effect on Zircaloy balloon and rupture performance. Here, this work used the BISON fuel performance code to assess the high-temperature creep model predictions with in situ data.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2025

Smart culture medium optimization for recombinant protein production: Experimental, modeling, and AI/ML-driven strategies

Recombinant protein production (RPP) is central to biotechnology, where recombinant proteins are used as either end products or catalysts in the synthesis of chemicals, fuels, and materials. Among the major cost drivers, culture medium plays a pivotal role in determining protein yield and quality. This review presents a comprehensive perspective on the critical stages of “smart” culture medium optimization: planning, screening, modeling, optimization, and validation. In the planning stage, we examine the nutritional and energetic roles of medium components, including carbon, nitrogen, amino acids, salts, and trace metals, and their impacts on culture parameters such as pH, oxidative state, and osmolality. We highlight the variability in trace metal content due to water sources, culture vessels, and raw materials, which can substantially influence RPP. The screening stage covers Design of Experiments (DoE) approaches, assessing their theoretical basis, implementation, and limitations. For modeling, we describe methods that integrate experimental data to develop predictive models for smart medium formulation. Model-based optimization strategies can then be employed to select optimal media compositions for a given application. The validation stage aims to evaluate model predictions and provide feedback for model training and refinement. Finally, we survey mechanistic and artificial intelligence/machine learning (AI/ML)-driven models as integrated, transformational tools for predictive modeling of bioprocess conditions, nutrient availability, cellular metabolism, and protein quality, with the goal of optimizing culture media to enhance protein yields while reducing costs and environmental impact. We conclude by addressing the challenges of translating laboratory-scale medium optimization to industrial-scale settings and exploring future AI/ML-driven approaches that may overcome current bottlenecks and accelerate medium design for RPP. Overall, this review provides a unified framework for advancing smart medium design in RPP.

Artificial Intelligence/Machine Learning (AI/ML)↗
DOE OSTI2023

Search for heavy resonances and quantum black holes in eμ, eτ, and μτ final states in proton-proton collisions at $\sqrt{s}$ = 13 TeV

A search is reported for heavy resonances and quantum black holes decaying into eμ, eτ, and μτ final states in proton-proton collision data recorded by the CMS experiment at the CERN LHC during 2016–2018 at $\sqrt{s}$ = 13 TeV, corresponding to an integrated luminosity of 138 fb -1 . The eμ, eτ, and μτ invariant mass spectra are reconstructed, and no evidence is found for physics beyond the standard model. Upper limits are set at 95% confidence level on the product of the cross section and branching fraction for lepton flavor violating signals. Three benchmark signals are studied: resonant τ sneutrino production in R parity violating supersymmetric models, heavy Z' gauge bosons with lepton flavor violating decays, and nonresonant quantum black hole production in models with extra spatial dimensions. Resonant τ sneutrinos are excluded for masses up to 4.2TeV in the eμ channel, 3.7TeV in the eτ channel, and 3.6TeV in the μτ channel. A Z' boson with lepton flavor violating couplings is excluded up to a mass of 5.0TeV in the eμ channel, up to 4.3Te V in the eτ channel, and up to 4.1TeV in the μτ channel. Quantum black holes in the benchmark model are excluded up to the threshold mass of 5.6TeV in the eμ channel, 5.2TeV in the eτ channel, and 5.0TeV in the μτ channel. In addition, model-independent limits are extracted to allow comparisons with other models for the same final states and similar event selection requirements. The results of these searches provide the most stringent limits available from collider experiments for heavy particles that undergo lepton flavor violating decays.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2022

A Fast Time-Stepping Strategy for Dynamical Systems Equipped with a Surrogate Model

Simulation of complex dynamical systems arising in many applications is computationally challenging due to their size and complexity. Model order reduction, machine learning, and other types of surrogate modeling techniques offer cheaper and simpler ways to describe the dynamics of these systems but are inexact and introduce additional approximation errors. In order to overcome the computational difficulties of the full complex models, on one hand, and the limitations of surrogate models, on the other, this work proposes a new accelerated time-stepping strategy that combines information from both. This approach is based on the multirate infinitesimal general-structure additive Runge--Kutta framework. The inexpensive surrogate model is integrated with a small time step to guide the solution trajectory, and the full model is treated with a large time step to occasionally correct for the surrogate model error and ensure convergence. Here, we provide a theoretical error analysis, and several numerical experiments, to show that this approach can be significantly more efficient than using only the full or only the surrogate model for the integration.

Surrogate models↗
DOE OSTI2022

Pyomo.DOE: An open-source package for model-based design of experiments in Python

Predictive mathematical models are a cornerstone of science and engineering. Yet selecting, calibrating, and validating said science-based models often remains an art in practice. Model-based design of experiments (MBDoE) provides a systematic framework to maximize information gain from experiments while minimizing time and resource costs. But MBDoE remains limited to niche application areas, in part because practitioners must integrate expertise in statistics, computational optimization, and modeling. To help reduce this barrier, we introduce Pyomo.DOE, an open-source package for MBDoE. Pyomo.DOE uses a nonlinear sensitivity analysis code k_aug to quickly approximate the Fisher information matrix and leverages a new stochastic programming abstraction. We demonstrate Pyomo.DOE with the first application of MBDoE to fixed-bed breakthrough experiments, which highlights the power of Pyomo.DOE to quantify the value of experimental modifications a priori for large-scale partial differential-algebraic equation (PDAE) models. Here we also provide a mathematical primer on MBDoE targeted at general chemical engineers.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2024

Ionic Pairs-Engineered Fluorinated Covalent Organic Frameworks Toward Direct Air Capture of CO 2

The covalent organic frameworks (COFs) possessing high crystallinity and capability to capture low-concentration CO 2 (400 ppm) from air are still underdeveloped. The challenge lies in simultaneously incorporating high-density active sites for CO 2 insertion and maintaining the ordered structure. Herein, a structure engineering approach is developed to afford an ionic pair-functionalized crystalline and stable fluorinated COF (F-COF) skeleton. The ordered structure of the F-COF is well maintained after the integration of abundant basic fluorinated alcoholate anions, as revealed by synchrotron X-ray scattering experiments. The breakthrough test demonstrates its attractive performance in capturing (400 ppm) CO 2 from gas mixtures via O$-$C bond formation, as indicated by the in situ spectroscopy and operando nuclear magnetic resonance spectroscopy using 13 C-labeled CO 2 sources. Both theoretical and experimental thermodynamic studies reveal the reaction enthalpy of ≈-40 kJ mol -1 between CO 2 and the COF scaffolds. This implies weaker interaction strength compared with state-of-the-art amine-derived sorbents, thus allowing complete CO 2 release with less energy input. The structure evolution study from synchrotron X-ray scattering and small-angle neutron scattering confirms the well-maintained crystalline patterns after CO 2 insertion. In conclusion, the as-developed proof-of-concept approach provides guidance on anchoring binding sites for direct air capture (DAC) of CO 2 in crystalline scaffolds.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2023

Search for excited $τ$-leptons and leptoquarks in the final state with $τ$-leptons and jets in pp collisions at $\sqrt{s}$ = 13 TeV with the ATLAS detector

A search is reported for excited $τ$-leptons and leptoquarks in events with two hadronically decaying $τ$-leptons and two or more jets. The search uses proton-proton ($pp$) collision data at $\sqrt{s}$ = 13 TeV recorded by the ATLAS experiment during the Run 2 of the Large Hadron Collider in 2015–2018. The total integrated luminosity is 139 fb -1 . The excited $τ$-lepton is assumed to be produced and to decay via a four-fermion contact interaction into an ordinary $τ$-lepton and a quark-antiquark pair. The leptoquarks are assumed to be produced in pairs via the strong interaction, and each leptoquark is assumed to couple to a charm or lighter quark and a $τ$-lepton. No excess over the background prediction is observed. Excited $τ$-leptons with masses below 2.8 TeV are excluded at 95% CL in scenarios with the contact interaction scale Λ set to 10 TeV. At the extreme limit of model validity where Λ is set equal to the excited $τ$-lepton mass, excited $τ$-leptons with masses below 4.6 TeV are excluded. Leptoquarks with masses below 1.3 TeV are excluded at 95% CL if their branching ratio to a charm quark and a $τ$-lepton equals 1. The analysis does not exploit flavour-tagging in the signal region.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2023

Search for resonant and non-resonant Higgs boson pair production in the $b\overline{b}{\tau}^{+}{\tau}^{-}$ decay channel using 13 TeV $pp$ collision data from the ATLAS detector

A search for Higgs boson pair production in events with two $b$-jets and two $τ$-leptons is presented, using a proton–proton collision dataset with an integrated luminosity of 139 fb -1 collected at $\sqrt{s}$ = 13 TeV by the ATLAS experiment at the LHC. Higgs boson pairs produced non-resonantly or in the decay of a narrow scalar resonance in the mass range from 251 to 1600 GeV are targeted. Events in which at least one $τ$-lepton decays hadronically are considered, and multivariate discriminants are used to reject the backgrounds. No significant excess of events above the expected background is observed in the non-resonant search. The largest excess in the resonant search is observed at a resonance mass of 1 TeV, with a local (global) significance of 3.1$σ$ (2.0$σ$). Observed (expected) 95% confidence-level upper limits are set on the non-resonant Higgs boson pair-production cross-section at 4.7 (3.9) times the Standard Model prediction, assuming Standard Model kinematics, and on the resonant Higgs boson pair-production cross-section at between 21 and 900 fb (12 and 840 fb), depending on the mass of the narrow scalar resonance.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2023

Amplitude analysis of the ${D}_s^{+}$ → π - π + π + decay

A Dalitz plot analysis of the ${D}_s^{+}$ → π - π + π + decay is presented. The analysis is based on proton-proton collision data recorded by the LHCb experiment at a centre of-mass energy of 8 TeV and corresponding to an integrated luminosity of 1.5 fb -1 . The resonant structure of the decay is obtained using a quasi-model-independent partial-wave analysis, in which the π + π - S-wave amplitude is parameterised as a generic complex function determined by a fit to the data. The S-wave component is found to be dominant, followed by the contribution from spin-2 resonances and a small contribution from spin-1 resonances. The latter includes the first observation of the ${D}_s^{+}$ → ω(782)π + channel in the ${D}_s^{+}$ → π - π + π + decay. The resonant structures of the ${D}_s^{+}$ → π - π + π + and ${D}^{+}$ → π - π + π + decays are compared, providing information about the mechanisms for the hadron formation in these decays.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2023

Measurement of CP asymmetries and branching fraction ratios of B – decays to two charm mesons

The CP asymmetries of seven B – decays to two charm mesons are measured using data corresponding to an integrated luminosity of 9 fb –1 of proton-proton collisions collected by the LHCb experiment. Decays involving a D *0 or ${D}_s^{\ast -}$ meson are analysed by reconstructing only the D 0 or ${D}_s^{-}$ decay products. This paper presents the first measurement of $\mathcal{A}$ CP (B – → ${D}_s^{\ast -}$ D 0 ) and $\mathcal{A}$ CP (B – → ${D}_s^{-}$D $\ast$0 ), and the most precise measurement of the other five CP asymmetries. There is no evidence of CP violation in any of the analysed decays. Additionally, two ratios between branching fractions of selected decays are measured.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2021

Updated search for $ {B}_c^{+} $ decays to two charm mesons

A data set corresponding to an integrated luminosity of 9 fb -1 of proton-proton collisions collected by the LHCb experiment has been analysed to search for B c + → D s ∗ + — D ∗ 0 decays. The decays are fully or partially reconstructed, where one or two missing neutral pions or photons from the decay of an excited charm meson are allowed. Upper limits for the branching fractions, normalised to B + decays to final states with similar topologies, are obtained for sixteen $ {B}_c^{+} $ decay modes. For the decay $ {B}_c^{+}\to {D}_s^{+}{\overline{D}}^0 $, an excess with a significance of 3.4 standard deviations is found.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2025

Development of a high-throughput method for processing sponge-stick samples to detect viable Bacillus anthracis spores

Since the national validation of the sponge-stick based method for detection of Bacillus anthracis spores in environmental samples, there have not been focused efforts to address the low throughput nature of the method, which processes only one sample at one time. Sample processing remains a serious bottleneck for rapidly analyzing large numbers of samples expected from a biological warfare attack. Therefore, we developed a high-throughput method to simultaneously process multiple sponge-stick samples to be better prepared for rapid response and recovery after wide area anthrax incidents. In this method, sponges are placed in 50 mL tubes containing 25 mL extraction buffer and shaken to release spores, after which the suspension is recovered for analysis. Here, we determined that an additional extraction step, conducted in the same tubes with 10 mL buffer, further increased spore recovery from sponge-stick by approximately 10 %. We determined that orbital shaking and multi-tube vortexing were both more effective than reciprocating shaking for recovering spores. We conducted simultaneous processing of up to 12 sponge-stick samples and demonstrated comparable spore recovery efficiencies to the traditional low-throughput stomacher-based method (approximately 60 % recovery at 10 2 -spore level and 75 % recovery at 10 4 -spore level for both methods in three replicate experiments, P > 0.05 for two-tailed t-tests for each experiment and spore level). We also demonstrated that our high-throughput method could be integrated with Rapid Viability-Polymerase Chain Reaction (RV-PCR) analysis and could detect levels as low as 40 spores per sponge even when challenged by a PCR particulate contaminant.

Anthrax↗
DOE OSTI2024

Girth and groomed radius of jets recoiling against isolated photons in lead-lead and proton-proton collisions at $\sqrt{s_{NN}} = 5.02$ TeV

This Letter presents the first measurements of the groomed jet radius R g and the jet girth g in events with an isolated photon recoiling against a jet in lead-lead (PbPb) and proton-proton (pp) collisions at the LHC at a nucleon-nucleon center-of-mass energy of 5.02 TeV. The observables R g and g provide a quantitative measure of how narrow or broad a jet is. The analysis uses PbPb and pp data samples with integrated luminosities of 1.7 nb -1 and 301 pb -1 , respectively, collected with the CMS experiment in 2018 and 2017. Events are required to have a photon with transverse momentum $p^{γ}_{T} > 100$ GeV and at least one jet back-to-back in azimuth with respect to the photon and with transverse momentum $p^{jet}_{T}$ such that $p^{jet}_{T}/p^{γ}_{T} > 0.4$ . The measured R g and g distributions are unfolded to the particle level, which facilitates the comparison between the PbPb and pp results and with theoretical predictions. It is found that jets with $p^{jet}_{T}/p^{γ}_{T} > 0.8$ , i.e., those that closely balance the photon $p^{γ}_{T}$ , are narrower in PbPb than in pp collisions. Relaxing the selection to include jets with $p^{jet}_{T}/p^{γ}_{T} > 0.4$ reduces the narrowing of the angular structure of jets in PbPb relative to the pp reference. This shows that selection bias effects associated with jet energy loss play an important role in the interpretation of jet substructure measurements.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2025

Spectroscopic Online Monitoring: Using a Multi-Track Visible Spectrometer to Facilitate a Mass Balance Study in a Simulated TALSPEAK Process

Nuclear energy is a promising low-carbon energy candidate to meet the increased demand for green energy, where the integration of fuel recycling can have significant benefits for material usage and waste reduction. Utilizing in situ monitoring tools can provide ample opportunities to better control and safeguard nuclear material recycle processes while also offering knowledge and insight into real-time solution properties. The simultaneous measurement of analytical targets in multiple process locations can enable real-time mass balance and material accountancy calculations. This is demonstrated here with a mass balance study of Nd 3+ on countercurrent aqueous/organic metal extraction within a single centrifugal contactor. The Nd 3+ concentration was simultaneously monitored at the inlets and outlets of both aqueous and organic phases using a visible absorbance detector that allowed for the simultaneous measurement of up to six locations. The Nd 3+ concentration was calculated by using chemical data science algorithms, where model training sets were collected on a single track of the detector. The discussion includes addressing the challenges of using a model collected on a single track and applying it as a model across the other tracks on the detector. Each track of the detector corresponds to one measurement location on the contactor. The difference in the integrated moles of Nd 3+ between the inlet and outlet at the end of the experiment was near zero, indicating that the mass balance of this experiment was maintained. Overall, the online spectroscopic monitoring was able to follow changing solution conditions and accurately measure the concentration of Nd 3+ in different locations within the contactor system.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2024

Numerical study of implosion instability mitigation in magnetically driven solid liner dynamic screw pinches

Magnetized liner inertial fusion experiments on the Z accelerator suffer from magneto-Rayleigh–Taylor instabilities (MRTI) that compromise integrity of the imploding cylindrical liner, limiting achievable fusion fuel conditions and ultimately reducing magneto-inertial fusion target performance. Dynamic screw pinches (DSP) provide a method to reduce MRTI in-flight via application of magnetic field line tension to the imploding liner outer surface. In contrast with z-pinches that drive implosions with an azimuthal magnetic field, dynamic screw pinches enforce an additional axial drive magnetic field component, making the overall drive magnetic field helical. As the liner implodes, cumulative MRTI development is reduced by dynamically shifting the orientation of the fastest growing instability modes. Three-dimensional magnetohydrodynamic simulations show that the DSP mechanism effectively stabilizes initially solid cylindrical liner implosions driven by Z-scale current pulses, indicating that MRTI mitigation increases with the ratio of axial to azimuthal drive magnetic field components (i.e., the drive field ratio). We also performed a spectral analysis of the simulated imploding density distributions, extracting wavelength and pitch angle of the simulated MRTI structures to study their dynamics during the implosion. Simulations of liners initially perturbed with drive-field-aligned sinusoidal structures indicate that MRTI mitigation in DSP implosions decreases with perturbation wavelength, once again suggestive of magnetic field line tension effects.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2026

Overview of the ASDEX Upgrade results

After a 26-month vent ASDEX Upgrade (AUG) went back in operation with a newly designed upper W-divertor suitable for alternative divertor configurations (featuring in-vessel coils and cryo-pump). Parameter scans and an extensive set of measurements were obtained and their interpretation is ongoing. Prompted by the ITER wall change, dedicated experiments on non-boronized plasma startup were contrasted to that employing asymmetric and more symmetric boronizations. The asymmetric boronization proved to be as beneficial as the more symmetric one, which is in contrast to previous model calculations assuming perfect sticking of boron (measurements suggest sticking ≈ 0.3). In the startup phase also the impurity influxes at the outboard limiters were investigated contrasting the unboronized case featuring cold edges (low- Z radiation) to the boronized case, in which the lifetime of the boron layers could be estimated. Pedestal stability investigations revealed that the quasi-continuous exhaust (QCE) regime is obtained when ballooning modes are active in the vicinity of the separatrix and the global peeling-ballooning stability is high enough. Thus, at high enough shaping and high gas flux both can be achieved and QCE is a consequence. The closely related enhanced D-alpha (EDA) mode is not clearly distinguishable from QCE, e.g. the quasi-coherent mode characteristic for EDA also shows up in QCE. In QCE the impurity transport is behaving benign as could be measured for Ne with a novel analysis method making use of a comprehensive set of CXRS measurements. For high radiative fractions the regime of the X-point radiator (XPR) is accessible at AUG and the understanding of its access conditions and behaviour is further developed. Due to the localized radiative cooling at the X-point the XPR can be well diagnosed and thus controlled. For negative triangularity shapes, further experiments at increased shaping resulted in strongly heated L-mode plasmas avoiding ELMs. Two integrated modelling approaches towards ITER suggested that core W-accumulation will be no issue for ITER and that the fusion yield in ITER may be Q = 12 (i.e. ITPA20-IL scaling is too pessimistic). Further, investigations of the ITER ramp-down in AUG provide insights into maintaining position control. Various aspects of shattered pellet injection were investigated in AUG and one of the results show that with increasing Ne fraction the radiation during the current quench increases and the current decay becomes faster.

ASDEX Upgrade↗
DOE OSTI2025

Impacts of benchmarking choices on inferred model skill of the Arctic–Boreal terrestrial carbon cycle

Abstract Land surface models require continuous validation against observations to improve and reduce simulation uncertainty. However, inferred model performance can be heavily influenced by subjective choices made in the selection and application of observational data products. A key area often misrepresented by models is the Arctic–Boreal region, which is a potential tipping point region in Earth’s climate system due to large permafrost carbon stocks that are vulnerable to release with climate warming. We use the International Land Model Benchmarking (ILAMB) framework to evaluate how the model skill of TRENDY-v9 models varies based on the choice of observational-based benchmark and how benchmarks are applied in model evaluation. This analysis uses global datasets integrated into ILAMB and new, regionally-specific observational products from the Arctic–Boreal Vulnerability Experiment. Our results cover the overall time period of 1979–2019 and show that model scores can vary substantially depending on the data product applied, with higher model scores indicating better model performance against observations. The lowest model scores occur when benchmarked against regional, compared to global, datasets. We also evaluate observed and modeled functional relationships between ecosystem respiration and air temperature and between gross primary production and precipitation. Here, we find that the magnitude and shape of the responses are strongly impacted by the choice of observational dataset and the approach used to construct the functional relationship benchmark. These results suggest that model evaluation studies could conclude a false sense of model skill if only using a single benchmark data product or if not applying regional data products when performing a regional model analysis. Collectively, our findings highlight the influence of benchmarking choices on model evaluation and point to the need for benchmarking guidelines when assessing model skill.

Poe, Jeralyn (ORCID:0000000318495278)↗