Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “model evaluation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,225 records · Page 68

Spring and summer contrast in new particle formation over nine forest areas in North America

Recent laboratory chamber studies indicate a significant role for highly oxidized low-volatility organics in new particle formation (NPF), but the actual role of these highly oxidized low-volatility organics in atmospheric NPF remains uncertain. Here, particle size distributions (PSDs) measured in nine forest areas in North America are used to characterize the occurrence and intensity of NPF and to evaluate model simulations using an empirical formulation in which formation rate is a function of the concentrations of sulfuric acid and low-volatility organics from alpha-pinene oxidation (Nucl-Org), and using an ion-mediated nucleation mechanism (excluding organics) (Nucl-IMN). On average, NPF occurred on ~ 70 % of days during March for the four forest sites with springtime PSD measurements, while NPF occurred on only ~ 10 % of days in July for all nine forest sites. Both Nucl-Org and Nucl-IMN schemes capture the observed high frequency of NPF in spring, but the Nucl-Org scheme significantly overpredicts while the Nucl-IMN scheme slightly underpredicts NPF and particle number concentrations in summer. Statistical analyses of observed and simulated ultrafine particle number concentrations and frequency of NPF events indicate that the scheme without organics agrees better overall with observations. The two schemes predict quite different nucleation rates (including their spatial patterns), concentrations of cloud condensation nuclei, and aerosol first indirect radiative forcing in North America, highlighting the need to reduce NPF uncertainties in regional and global earth system models.

54 ENVIRONMENTAL SCIENCES↗

Environmental effects on aerosol–cloud interaction in non-precipitating marine boundary layer (MBL) clouds over the eastern North Atlantic

Abstract. Over the eastern North Atlantic (ENA) ocean, a total of 20 non-precipitating single-layer marine boundary layer (MBL) stratus and stratocumulus cloud cases are selected to investigate the impacts of the environmental variables on the aerosol–cloud interaction (ACIr) using the ground-based measurements from the Department of Energy Atmospheric Radiation Measurement (ARM) facility at the ENA site during 2016–2018. The ACIr represents the relative change in cloud droplet effective radius re with respect to the relative change in cloud condensation nuclei (CCN) number concentration at 0.2 % supersaturation (NCCN,0.2 %) in the stratified water vapor environment. The ACIr values vary from −0.01 to 0.22 with increasing sub-cloud boundary layer precipitable water vapor (PWVBL) conditions, indicating that re is more sensitive to the CCN loading under sufficient water vapor supply, owing to the combined effect of enhanced condensational growth and coalescence processes associated with higher Nc and PWVBL. The principal component analysis shows that the most pronounced pattern during the selected cases is the co-variations in the MBL conditions characterized by the vertical component of turbulence kinetic energy (TKEw), the decoupling index (Di), and PWVBL. The environmental effects on ACIr emerge after the data are stratified into different TKEw regimes. The ACIr values, under both lower and higher PWVBL conditions, more than double from the low-TKEw to high-TKEw regime. This can be explained by the fact that stronger boundary layer turbulence maintains a well-mixed MBL, strengthening the connection between cloud microphysical properties and the below-cloud CCN and moisture sources. With sufficient water vapor and low CCN loading, the active coalescence process broadens the cloud droplet size spectra and consequently results in an enlargement of re. The enhanced activation of CCN and the cloud droplet condensational growth induced by the higher below-cloud CCN loading can effectively decrease re, which jointly presents as the increased ACIr. This study examines the importance of environmental effects on the ACIr assessments and provides observational constraints to future model evaluations of aerosol–cloud interactions.

54 ENVIRONMENTAL SCIENCES↗

Application of PRIM for understanding patterns in carbon dioxide model-observation differences

Reducing uncertainties in regional carbon balances requires a better understanding of CO 2 transport in synoptic weather systems. Here, we apply the Patient Rule Induction Method (PRIM), a data-mining method to identify high-density regions for a target-class within an input parameter space, to airborne observations of potential temperature, wind speed, water vapor mixing ratio, and CO 2 dry mol fraction gathered during the Atmospheric Carbon and Transport (ACT)-America Summer 2016 and Winter 2017 campaigns. ACT observations were targeted at expert-designated cases of fair weather and near-frontal warm and cold sector air at atmospheric boundary-layer, lower-, and higher free tropospheric levels (ABL, LFT, and HFT, respectively). We investigate atmospheric characteristics of these pre-defined cases and associated CO 2 model-observation-differences in the mesoscale WRF-Chem model. PRIM results separate winter- and summertime observations as well as observations from ABL, LFT, and HFT with enrichment factors of 4.0–20.5 inside the PRIM box compared to the entire dataset but cannot distinguish between near-frontal warm and cold sector observations in the higher free troposphere. Analyzing of the parameter space constrained by PRIM, we find that large magnitude model observation differences preferentially associated with times when atmospheric conditions are less typical. This association suggests that PRIM could provide a useful tool for isolating atmospheric conditions with large-magnitude and non-Gaussian CO 2 -residuals for targeted transport model evaluation and to potentially improve inversion results during synoptically active periods.

Gerken, Tobias [James Madison Univ., Harrisonburg,↗

Characterization of shallow oceanic precipitation using profiling and scanning radar observations at the Eastern North Atlantic ARM observatory

Shallow oceanic precipitation variability is documented using three second-generation radar systems located at the Atmospheric Radiation Measurement (ARM) Eastern North Atlantic observatory: ARM zenith radar (KAZR2), the Ka-band scanning ARM cloud radar (KaSACR2) and the X-band scanning ARM precipitation radar (XSAPR2). First, the radar systems and measurement post-processing techniques, including sea-clutter removal and calibration against colocated disdrometer and Global Precipitation Mission (GPM) observations are described. Then, we present how a combination of profiling radar and lidar observations can be used to estimate adaptive (in both time and height) parameters that relate radar reflectivity (Z) to precipitation rate (R) in the form Z=αR β , which we use to estimate precipitation rate over the domain observed by XSAPR2. Furthermore, constant altitude plan position indicator (CAPPI) gridded XSAPR2 precipitation rate maps are also constructed. Hourly precipitation rate statistics estimated from the three radar systems differ because KAZR2 is more sensitive to shallow virga and XSAPR2 suffers from less attenuation than KaSACR2 and as such is best suited for characterizing intermittent and mesoscale-organized precipitation. Further analysis reveals that precipitation rate statistics obtained by averaging 12 h of KAZR2 observations can be used to approximate that of a 40 km radius domain averaged over similar time periods. However, it was determined that KAZR2 is unsuitable for characterizing domain-averaged precipitation rate over shorter periods. But even more fundamentally, these results suggest that these observations cannot produce an objective domain precipitation estimate and that the simultaneous use of forward simulators is desirable to guide model evaluation studies.

54 ENVIRONMENTAL SCIENCES↗

HadISDH land surface multi-variable humidity and temperature record for climate monitoring

HadISDH.2.0.0 is the first gridded, multi-variable humidity and temperature in situ observations-only climate-data product that is homogenised and annually updated. It provides physically consistent estimates for specific humidity, vapour pressure, relative humidity, dew point temperature, wet bulb temperature, dew point depression and temperature. It is a monthly mean gridded (5° by 5°) product with uncertainty estimates that account for spatio-temporal sampling, climatology calculation, homogenisation and irreducible random measurement effects. It provides a tool for the long-term monitoring of a variety of humidity-related variables which have different impacts and implications for society. It is also useful for climate model evaluation and reanalyses validation. HadISDH.2.0.0 is shown to be in good agreement both with other estimates and with theoretical understanding. The data set is available from 1973 to the present. The theme common to all variables is of a warming world with more water vapour present in the atmosphere. The largest increases in water vapour are found over the tropics and the Mediterranean. Over the tropics and high northern latitudes the surface air over land is becoming more saturated. However, despite increasing water vapour over the mid-latitudes and Mediterranean, the surface air over land is becoming less saturated. These observed features may be due to atmospheric circulation changes, land–sea warming disparities and reduced water availability or changed land surface properties.

54 ENVIRONMENTAL SCIENCES↗

Radiocarbon measurements of ecosystem respiration and soil pore-space CO 2 in Utqiagvik (Barrow), Alaska

Radiocarbon measurements of ecosystem respiration and soil pore space CO 2 are useful for determining the sources of ecosystem respiration,identifying environmental controls on soil carbon cycling rates, and parameterizing and evaluating models of the carbon cycle. We measured flux rates and radiocarbon content of ecosystem respiration, as well as radiocarbon in soil profile CO 2 in Utqiagvik (Barrow), Alaska, during the summers of 2012, 2013, and 2014.We found that radiocarbon in ecosystem respiration(Δ 14 C Reco ) ranged from +60.5 to –160‰ with a median value of +23.3‰. Ecosystem respiration became more depleted in radiocarbon from summer to autumn, indicating increased decomposition of old soil organic carbon and/or decreased CO 2 production from fast-cycling carbon pools. Across permafrost features,ecosystem respiration from high-centered polygons was depleted in radiocarbon relative to other polygon types. Radiocarbon content in soil pore-space CO 2 varied between –7.1 and –280‰, becoming more negative with depth in individual soil profiles. These pore-space radiocarbon values correspond to CO 2 mean ages of 410 to 3350 years, based on a steady-state, one-pool model. Together, these data indicate that deep soil respiration was derived primarily from old, slow-cycling carbon, but that total CO 2 fluxes depended largely on autotrophic respiration and heterotrophic decomposition of fast-cycling carbon within the shallowest soil layers. The relative contributions of these different CO 2 sources were highly variable across microtopographic features and time in the sampling season. The highly negative Δ 14 C values in soil pore-space CO 2 and autumn ecosystem respiration indicate that when it is not frozen, very old soil carbon is vulnerable to decomposition.

54 ENVIRONMENTAL SCIENCES↗

Atmospheric observations made at Oliktok Point, Alaska, as part of the Profiling at Oliktok Point to Enhance YOPP Experiments (POPEYE) campaign

Between 1 July and 30 September 2018, small unmanned aircraft systems (sUAS), tethered balloon systems (TBSs), and additional radiosondes were deployed at Oliktok Point, Alaska, to measure the atmosphere in support of the second special observing period for the Year of Polar Prediction (YOPP). These measurements, collected as part of the Profiling at Oliktok Point to Enhance YOPP Experiments (POPEYE) campaign, targeted quantities related to enhancing our understanding of boundary layer structure, cloud and aerosol properties and surface–atmosphere exchange and providing extra information for model evaluation and improvement work. Over the 3-month campaign, a total of 59 DataHawk2 sUAS flights, 52 TBS flights, and 238 radiosonde launches were completed as part of POPEYE. The data from these coordinated activities provide a comprehensive three-dimensional data set of the atmospheric state (air temperature, humidity, pressure, and wind), surface skin temperature, aerosol properties, and cloud microphysical information over Oliktok Point.

54 ENVIRONMENTAL SCIENCES↗

Measurements from mobile surface vehicles during the Lower Atmospheric Profiling Studies at Elevation – a Remotely-piloted Aircraft Team Experiment (LAPSE-RATE)

Abstract. Between 14 and 20 July 2018, small unmanned aircraft systems (UASs) were deployed to the San Luis Valley of Colorado (USA) alongside surface-based remote sensors, in situ sensors, and radiosonde systems as part of the Lower Atmospheric Profiling Studies at Elevation – a Remotely-piloted Aircraft Team Experiment (LAPSE-RATE). The measurements collected as part of LAPSE-RATE targeted quantities related to enhancing our understanding of boundary layer structure, cloud and aerosol properties and surface–atmosphere exchange and provide detailed information to support model evaluation and improvement work. Additionally, intensive intercomparison between the different unmanned aircraft platforms was completed. The current paper describes the observations obtained using three different types of surface-based mobile observing vehicles. These included the University of Colorado Mobile UAS Research Collaboratory (MURC), the National Oceanic and Atmospheric Administration National Severe Storms Laboratory Mobile Mesonet, and two University of Nebraska Combined Mesonet and Tracker (CoMeT) vehicles. Over the 1-week campaign, a total of 143 h of data were collected using this combination of vehicles. The data from these coordinated activities provide detailed perspectives on the spatial variability of atmospheric state parameters (air temperature, humidity, pressure, and wind) throughout the northern half of the San Luis Valley. These datasets have been checked for quality and published to the Zenodo data archive under a specific “community” setup for LAPSE-RATE (https://zenodo.org/communities/lapse-rate/, last access: 21 January 2021) and are accessible at no cost by all registered users. The primary dataset DOIs are https://doi.org/10.5281/zenodo.3814765 (CU MURC measurements; de Boer et al., 2020d), https://doi.org/10.5281/zenodo.3738175 (NSSL MM measurements; Waugh, 2020), and https://doi.org/10.5281/zenodo.3838724 (UNL CoMeT measurements; Houston and Erwin, 2020).

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

University of Colorado and Black Swift Technologies RPAS-based measurements of the lower atmosphere during LAPSE-RATE

Abstract. Between 14 and 20 July 2018, small remotely piloted aircraft systems (RPASs) were deployed to the San Luis Valley of Colorado (USA) together with a variety of surface-based remote and in situ sensors as well as radiosonde systems as part of the Lower Atmospheric Profiling Studies at Elevation – a Remotely-piloted Aircraft Team Experiment (LAPSE-RATE). The observations from LAPSE-RATE were aimed at improving our understanding of boundary layer structure, cloud and aerosol properties, and surface–atmosphere exchange and provide detailed information to support model evaluation and improvement work. The current paper describes the observations obtained using four different types of RPASs deployed by the University of Colorado Boulder and Black Swift Technologies. These included the DataHawk2, the Talon and the TTwistor (University of Colorado), and the S1 (Black Swift Technologies). Together, these aircraft collected over 30 h of data throughout the northern half of the San Luis Valley, sampling altitudes between the surface and 914 m a.g.l. Data from these platforms are publicly available through the Zenodo archive and are co-located with other LAPSE-RATE data as part of the Zenodo LAPSE-RATE community (https://zenodo.org/communities/lapse-rate/, last access: 27 May 2021). The primary DOIs for these datasets are https://doi.org/10.5281/zenodo.3891620 (DataHawk2, de Boer et al., 2020a, e), https://doi.org/10.5281/zenodo.4096451 (Talon, de Boer et al., 2020d), https://doi.org/10.5281/zenodo.4110626 (TTwistor, de Boer et al., 2020b), and https://doi.org/10.5281/zenodo.3861831 (S1, Elston and Stachura, 2020).

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Automatic tuning of the Community Atmospheric Model (CAM5) by using short-term hindcasts with an improved downhill simplex optimization method

Traditional trial-and-error tuning of uncertain parameters in global atmospheric general circulation models (GCMs) is time consuming and subjective. This study explores the feasibility of automatic optimization of GCM parameters for fast physics by using short-term hindcasts. An automatic workflow is described and applied to the Community Atmospheric Model (CAM5) to optimize several parameters in its cloud and convective parameterizations. We show that the auto-optimization leads to 10 % reduction of the overall bias in CAM5, which is already a well-calibrated model, based on a predefined metric that includes precipitation, temperature, humidity, and longwave/shortwave cloud forcing. The computational cost of the entire optimization procedure is about equivalent to a single 12-year atmospheric model simulation. The tuning reduces the large underestimation in the CAM5 longwave cloud forcing by decreasing the threshold relative humidity and the sedimentation velocity of ice crystals in the cloud schemes; it reduces the overestimation of precipitation by increasing the adjustment time in the convection scheme. The physical processes behind the tuned model performance for each targeted field are discussed. Limitations of the automatic tuning are described, including the slight deterioration in some targeted fields that reflect the structural errors of the model. It is pointed out that automatic tuning can be a viable supplement to process-oriented model evaluations and improvement.

54 ENVIRONMENTAL SCIENCES↗

PyFLEXTRKR: a flexible feature tracking Python software for convective cloud analysis

Abstract. This paper describes the new open-source framework PyFLEXTRKR (Python FLEXible object TRacKeR), a flexible atmospheric feature tracking software package with specific capabilities to track convective clouds from a variety of observations and model simulations. This software can track any atmospheric 2D objects and handle merging and splitting explicitly. The package has a collection of multi-object identification algorithms, scalable parallelization options, and has been optimized for large datasets including global high-resolution data. We demonstrate applications of PyFLEXTRKR on tracking individual deep convective cells and mesoscale convective systems from observations and model simulations ranging from large-eddy resolving (∼100s m) to mesoscale (∼10s km) resolutions. Visualization, post-processing, and statistical analysis tools are included in the package. New Lagrangian analyses of convective clouds produced by PyFLEXTRKR applicable to a wide range of datasets and scales facilitate advanced model evaluation and development efforts as well as scientific discovery.

54 ENVIRONMENTAL SCIENCES↗

Automatic tuning of the Community Atmospheric Model CAM5.3 by using short-term hindcasts with an improved downhill simplex optimization method

Traditional trial-and-error tuning of uncertain parameters in global atmospheric General Circulation Models (GCM) is time consuming and subjective. This study explores the feasibility of automatic optimization of GCM parameters for fast physics by using short-term hindcasts. An automatic workflow is described and applied to the Community Atmospheric Model (CAM5) to optimize several parameters in its cloud and convective parameterizations. We show that the auto-optimization leads to 10 % reduction of the overall bias in CAM5, which is already a well calibrated model, based on a pre-defined metric that includes precipitation, temperature, humidity, and longwave/shortwave cloud forcing. The computational cost of the entire optimization procedure is about equivalent to about a single 12-year atmospheric model simulation. The tuning reduces the large underestimation in the CAM5 longwave cloud forcing by decreasing the threshold relative humidity and the sedimentation velocity of ice crystals in the cloud schemes; it reduces the overestimation of precipitation by increasing the adjustment time in the convection scheme. The physical processes behind the tuned model performance for each targeted field are discussed. Limitations of the automatic tuning are described, including the slight deterioration in some targeted fields that reflect the structural errors of the model. It is pointed out that automatic tuning can be a viable supplement to process-oriented model evaluations and improvement.

54 ENVIRONMENTAL SCIENCES↗

Derived Optimal Linear Combination Evapotranspiration (DOLCE): a global gridded synthesis ET estimate

Accurate global gridded estimates of evapotranspiration (ET) are key to understanding water and energy budgets, in addition to being required for model evaluation. Several gridded ET products have already been developed which differ in their data requirements, the approaches used to derive them and their estimates, yet it is not clear which provides the most reliable estimates. This paper presents a new global ET dataset and associated uncertainty with monthly temporal resolution for 2000–2009. Six existing gridded ET products are combined using a weighting approach trained by observational datasets from 159 FLUXNET sites. The weighting method is based on a technique that provides an analytically optimal linear combination of ET products compared to site data and accounts for both the performance differences and error covariance between the participating ET products. We examine the performance of the weighting approach in several in-sample and out-of-sample tests that confirm that point-based estimates of flux towers provide information on the grid scale of these products. We also provide evidence that the weighted product performs better than its six constituent ET product members in four common metrics. Uncertainty in the ET estimate is derived by rescaling the spread of participating ET products so that their spread reflects the ability of the weighted mean estimate to match flux tower data. While issues in observational data and any common biases in participating ET datasets are limitations to the success of this approach, future datasets can easily be incorporated and enhance the derived product.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

A total of 19 months of daily weather logging on the US east coast: the WFIP3 event log

The Third Wind Forecast Improvement Project (WFIP3) is a multi-institutional field campaign designed to advance the understanding and prediction of the offshore atmospheric boundary layer along the US east coast. Extending from February 2024 through August 2025, WFIP3 combines long-term coastal and offshore measurements with targeted modeling and forecasting efforts. This data paper presents the WFIP3 event log, a curated record of 578 d of meteorological phenomena and field observations that complements the campaign's extensive high-frequency datasets. The event log provides both manually documented daily weather discussions and automatically derived indicators of atmospheric processes – including low-level jets, wind ramps, extreme wind veer, and weak wind conditions – based on observations from scanning lidars deployed at three coastal and offshore sites. The dataset offers structured metadata, standardized time and site identifiers, and consistent terminology to facilitate its integration with WFIP3's observational and modeling data products. The log supports diverse applications, from model evaluation and forecast verification to the selection of case studies on offshore boundary-layer dynamics. The WFIP3 event log is publicly available through the US Department of Energy's Wind Data Hub, providing the research community with a transparent and enduring contextual reference for the interpretation and use of WFIP3 measurements.

17 WIND ENERGY↗

Multifidelity multiobjective optimization for wake-steering strategies

Abstract. Wake steering is an emerging wind power plant control strategy where upstream turbines are intentionally yawed out of perpendicular alignment with the incoming wind, thereby “steering” wakes away from downstream turbines. However, trade-offs between the gains in power production and fatigue loads induced by this control strategy are the subject of continuing investigation. In this study, we present a multifidelity multiobjective optimization approach for exploring the Pareto front of trade-offs between power and loading during wake steering. A large eddy simulation is used as the high-fidelity model, where an actuator line representation is used to model wind turbine blades and a rainflow-counting algorithm is used to compute damage equivalent loads. A coarser simulation with a simpler loads model is employed as a supplementary low-fidelity model. Multifidelity Bayesian optimization is performed to iteratively learn both a surrogate of the low-fidelity model and an additive discrepancy function, which maps the low-fidelity model to the high-fidelity model. Each optimization uses the expected hypervolume improvement acquisition function, weighted by the total cost of a proposed model evaluation in the multifidelity case. The multifidelity approach is able to capture the logit function shape of the Pareto frontier at a computational cost only 30 % that of the single-fidelity approach. Additionally, we provide physical insights into the vortical structures in the wake that contribute to the Pareto front shape.

17 WIND ENERGY↗

Climate Model Output Rewriter

The Climate Model Output Rewriter (CMOR) software was first developed by LLNL’s PCMDI program in early 2000s and was formally released with v1.0 (July 2006), v2.0 (January 2011), and v3.1(June 2016). CMOR is used to produce Climate and Forecast Convention (http://cfconventions.org/) CF-compliant netCDF files, in the standard format required to satisfy the World Climate Research Program (WCRP) Coupled Model Intercomparison Project (CMIP). The software has been used across multiple phases of the Earth System Modeling (ESM) project CMIP (CMIP3, CMIP5, CMIP6, and planned use in CMIP7) along with numerous parallel projects focused on preparation observations for use in model evaluation (obs4MIPs) and forcing datasets (input4MIPs) to guide ESM simulations to meet strict experimental protocols. More information can be obtained from the CMOR website and code repositories: https://cmor.llnl.gov/; https://github.com/pcmdi/cmor; https://github.com/PCMDI/cmor3_documentation The ESM variable definitions used as input for CMOR can also be viewed in code repositories: https://github.com/PCMDI/cmip3-cmor-tables/; https://github.com/PCMDI/cmip5-cmor-tables/; https://github.com/PCMDI/cmip6-cmor-tables/

Mauzey, ChristopherF↗

three-dimensional constrained variational analysis

This is a dataset from three-dimensional constrained variational analysis (3DCVA). It can be used to generate large-scale forcing data for SCM/CRM/LES, or evaluate model results.

54 ENVIRONMENTAL SCIENCES↗

Predictive performance of multi-model ensemble forecasts of COVID-19 across European nations

Background: Short-term forecasts of infectious disease burden can contribute to situational awareness and aid capacity planning. Based on best practice in other fields and recent insights in infectious disease epidemiology, one can maximise the predictive performance of such forecasts if multiple models are combined into an ensemble. Here, we report on the performance of ensembles in predicting COVID-19 cases and deaths across Europe between 08 March 2021 and 07 March 2022. Methods: We used open-source tools to develop a public European COVID-19 Forecast Hub. We invited groups globally to contribute weekly forecasts for COVID-19 cases and deaths reported by a standardised source for 32 countries over the next 1–4 weeks. Teams submitted forecasts from March 2021 using standardised quantiles of the predictive distribution. Each week we created an ensemble forecast, where each predictive quantile was calculated as the equally-weighted average (initially the mean and then from 26th July the median) of all individual models’ predictive quantiles. We measured the performance of each model using the relative Weighted Interval Score (WIS), comparing models’ forecast accuracy relative to all other models. We retrospectively explored alternative methods for ensemble forecasts, including weighted averages based on models’ past predictive performance. Results: Over 52 weeks, we collected forecasts from 48 unique models. We evaluated 29 models’ forecast scores in comparison to the ensemble model. We found a weekly ensemble had a consistently strong performance across countries over time. Across all horizons and locations, the ensemble performed better on relative WIS than 83% of participating models’ forecasts of incident cases (with a total N=886 predictions from 23 unique models), and 91% of participating models’ forecasts of deaths (N=763 predictions from 20 models). Across a 1–4 week time horizon, ensemble performance declined with longer forecast periods when forecasting cases, but remained stable over 4 weeks for incident death forecasts. In every forecast across 32 countries, the ensemble outperformed most contributing models when forecasting either cases or deaths, frequently outperforming all of its individual component models. Among several choices of ensemble methods we found that the most influential and best choice was to use a median average of models instead of using the mean, regardless of methods of weighting component forecast models. Conclusions: Our results support the use of combining forecasts from individual models into an ensemble in order to improve predictive performance across epidemiological targets and populations during infectious disease epidemics. Our findings further suggest that median ensemble methods yield better predictive performance more than ones based on means. Our findings also highlight that forecast consumers should place more weight on incident death forecasts than incident case forecasts at forecast horizons greater than 2 weeks. Funding: AA, BH, BL, LWa, MMa, PP, SV funded by National Institutes of Health (NIH) Grant 1R01GM109718, NSF BIG DATA Grant IIS-1633028, NSF Grant No.: OAC-1916805, NSF Expeditions in Computing Grant CCF-1918656, CCF-1917819, NSF RAPID CNS-2028004, NSF RAPID OAC-2027541, US Centers for Disease Control and Prevention 75D30119C05935, a grant from Google, University of Virginia Strategic Investment Fund award number SIF160, Defense Threat Reduction Agency (DTRA) under Contract No. HDTRA1-19-D-0007, and respectively Virginia Dept of Health Grant VDH-21-501-0141, VDH-21-501-0143, VDH-21-501-0147, VDH-21-501-0145, VDH-21-501-0146, VDH-21-501-0142, VDH-21-501-0148. AF, AMa, GL funded by SMIGE - Modelli statistici inferenziali per governare l'epidemia, FISR 2020-Covid-19 I Fase, FISR2020IP-00156, Codice Progetto: PRJ-0695. AM, BK, FD, FR, JK, JN, JZ, KN, MG, MR, MS, RB funded by Ministry of Science and Higher Education of Poland with grant 28/WFSN/2021 to the University of Warsaw. BRe, CPe, JLAz funded by Ministerio de Sanidad/ISCIII. BT, PG funded by PERISCOPE European H2020 project, contract number 101016233. CP, DL, EA, MC, SA funded by European Commission - Directorate-General for Communications Networks, Content and Technology through the contract LC-01485746, and Ministerio de Ciencia, Innovacion y Universidades and FEDER, with the project PGC2018-095456-B-I00. DE., MGu funded by Spanish Ministry of Health / REACT-UE (FEDER). DO, GF, IMi, LC funded by Laboratory Directed Research and Development program of Los Alamos National Laboratory (LANL) under project number 20200700ER. DS, ELR, GG, NGR, NW, YW funded by National Institutes of General Medical Sciences (R35GM119582; the content is solely the responsibility of the authors and does not necessarily represent the official views of NIGMS or the National Institutes of Health). FB, FP funded by InPresa, Lombardy Region, Italy. HG, KS funded by European Centre for Disease Prevention and Control. IV funded by Agencia de Qualitat i Avaluacio Sanitaries de Catalunya (AQuAS) through contract 2021-021OE. JDe, SMo, VP funded by Netzwerk Universitatsmedizin (NUM) project egePan (01KX2021). JPB, SH, TH funded by Federal Ministry of Education and Research (BMBF; grant 05M18SIA). KH, MSc, YKh funded by Project SaxoCOV, funded by the German Free State of Saxony. Presentation of data, model results and simulations also funded by the NFDI4Health Task Force COVID-19 ( https://www.nfdi4health.de/task-force-covid-19-2 ) within the framework of a DFG-project (LO-342/17-1). LP, VE funded by Mathematical and Statistical modelling project (MUNI/A/1615/2020), Online platform for real-time monitoring, analysis and management of epidemic situations (MUNI/11/02202001/2020); VE also supported by RECETOX research infrastructure (Ministry of Education, Youth and Sports of the Czech Republic: LM2018121), the CETOCOEN EXCELLENCE (CZ.02.1.01/0.0/0.0/17-043/0009632), RECETOX RI project (CZ.02.1.01/0.0/0.0/16-013/0001761). NIB funded by Health Protection Research Unit (grant code NIHR200908). SAb, SF funded by Wellcome Trust (210758/Z/18/Z).

60 APPLIED LIFE SCIENCES↗