Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Probabilistic Forecast”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5

A model of subjective probabilities from small groups

Methods for aggregating the opinions of individual forecasters in order to improve the quality of probabilistic forecasts are presented. Experimental results obtained by Seaver in his study of probability judgments by groups of four people are considered. The decision variable partition model of subjective probability and a simple model of the effects of interaction on judgments were used to simulate the group judgment of discrete probabilities investigated by Seaver. The initial results of the simulation are very promising in that (1) they show the principal effects Seaver observed and (2) these effects can, for the most part, be traced to specific characteristics of the models.

Ferrell, W. R.↗

A Machine Learning Approach to Improve Air Traffic Management Initiatives

Collaborating closely with commercial air carriers and related organizations, the Federal Aviation Administration(FAA) regulates air traffic and ensures the safety and efficiency of air operations. Air traffic controllers make strategic decisions, such as delaying, rerouting, or canceling flights, partly based on guidance provided by the FAA’s Air TrafficControl System Command Center (ATCSCC). The guidance includes, among other things, control measures known asTraffic Management Initiatives (TMIs) designed to enhance safety and improve operational efficiency. TMIs play a crucial role in managing the demand and capacity within the U.S. National Airspace System (NAS). Two major TMIs that are routinely used (primarily to mitigate the adverse effects of bad weather) are Ground Delay Programs (GDPs) andGround Stops (GSs). In a GDP, flights destined for airports facing thunderstorm activity experience delays at their origin airports. This proactive approach minimizes the risk of routing aircraft through hazardous weather conditions and also replaces (fuel burning) airborne delays with ground delays. In a GS, a temporary restriction is imposed on the departure or arrival of aircraft at a specific airport or within a designated airspace. Although other TMIs (e.g., miles-in-trail) are also implemented as part of (air) traffic flow management in the NAS, the focus of this work is on GDPs and GSs. Since TMIs, by design, lead to flight delays or cancellations, it is crucial to put in place the right set of parameters(e.g., scope and duration of the GDP). For example, when the end time of a GDP extends beyond what is necessary, it imposes unnecessary delays on departing flights. This situation could occur as a result of inaccurate prediction of the(required) duration of the GDP based on the weather forecast. On the other hand, if a GDP ends prematurely before the underlying capacity constraints are resolved at the destination airport, it may result in airborne holding. The delicate balance lies in matching the termination of the GDP precisely with the resolution of capacity constraints, avoiding both the imposition of unnecessary ground delays and the need for airborne holding due to premature program termination.Failing to specify the right parameters for TMIs also leads to flight delays, creating a significant obstacle in managing the increasing traffic volumes causing increased work load for the controllers. To address this issue, we propose the integration of Machine Learning (ML) models in the traffic flow management(TFM) pipeline. In current operations, decisions are made by human experts based on extensive training, historical patterns, available traffic and weather data. Since we have an abundance of data from past events that tell us the likely impact of various TMIs, by ingesting historical data, properly trained ML models can offer valuable insights and aid human decision-making. With the FAA increasingly exploring advanced analytics, ML emerges as a focal point for enhancing TFM within the National Airspace System (NAS). As a first step, this study aims to provide traffic controllers with decision-making support for the issuance and adjustment of TMIs. Data analytics and machine learning have been previously employed to address some of the challenges associated with TMIs. Numerous studies have concentrated on various facets of TMI issuance, exploring factors influencing TMI parameters, including arrival rate, airport capacity, and delay prediction. For example, using weather forecasts, several statistical methods were used to produce probabilistic capacity profiles which in conjunction with deterministic models provided insights into the GDP planning process [1–4]. The downside of using deterministic models is that they rely on fixed inputs and predetermined rules, which lack the ability to account for the inherent uncertainty and variability present in real-world scenarios. In a separate series of studies, researchers aimed to predict the occurrences of GDPs and GSs. The majority of these studies utilized various supervised learning methods, including Decision Trees, Naive Bayes, Support VectorMachines, and Random Forests to analyze the influence of weather conditions and arrival demand on TMI incidents[5–8]. However, these studies primarily focused on predicting the incidence of TMIs without explicitly addressing the scope of TMIs, including their duration and their geographical coverage. Furthermore, the emphasis of these studies was largely on GDPs, given their higher frequency and longer duration when compared to GSs. A limited number of studies focused on predicting the parameters of TMIs, specifically addressing their duration and extent. In one such study focusing on optimizing the TMI parameters at San Francisco International Airport (SFO),the authors utilized a probabilistic forecast of fog [9]. They simulated various capacity scenarios based on the (fog)burn-off forecasts, selecting GDP parameters that minimized airborne and overall ground delays. However, this approach exclusively emphasizes stratus (fog) burn-off as the primary determinant of GDP and GS, neglecting other influential factors like severe weather events, runway closures, lower capacity than traffic demand, and other important variables. Given the complexity of predicting the TMI and determining its scope, we seek a more holistic approach. We aim to consider all significant factors that could impact TMIs and their parameters. What sets this research apart is the fusion of all data sources relevant to the issuance and adjustment of TMIs and it represents the first comprehensive attempt to optimize TMIs in this manner. Since this comprehensive solution involves various aspects, we break down the problem into smaller components and input all parameters into a unified model called the “TMI Adjuster”. Figure 1 shows the overall framework and the list of datasets used in each model. The objective of the TMI Adjuster module is to deliver reliable, consistent and expedited recommendations for the progression, adjustment, and termination of TMIs. The ML solution entails developing a pipeline capable of predicting the necessity of a TMI (e.g., GS or GDP) along with its various parameters. For example, in the case of a GS, this includes the scope of the GS either in terms of distance from the destination airport or based on pre-defined airspace sectors. Here, scope refers to those regions and departing airports that are subject to the GS. In this paper, we concentrate on the issuance of GSs in the three major airports in the New York area — LaGuardia(LGA), John F. Kennedy International (JFK), and Newark Liberty International (EWR). We fuse traffic, weather and other relevant aviation data from years 2017 to 2019 to train and validate the ML models. In particular, we use the following datasets: •Terminal Aerodrome Forecast (TAF): meteorological forecasts specific to each airport, issued four times a day, covering predefined time periods. •TMI data: includes all GSs and GDPs along with their respective parameters. •Aviation System Performance Metrics (ASPM): includes traffic related data such as aircraft delays, arrival, and departure rates. •Notices to Airmen (NOTAMs): utilized to extract runway closure data and manage interdependencies between terminals in close proximity. •Flight cancellation data •Airspace Flow Programs (AFP): includes information on flight airborne holdings caused by TMIs. The data preprocessing entails transforming ASPM, TMI, AFP, NOTAMs, and weather data into an hourly format and consolidating all datasets by merging them based on date and time as the primary key. The TMI Adjuster framework comprises two parallel models: one dedicated to GS and a second model focused on GDP. As previously mentioned, our specific focus is on the GS model as a multi-classification problem. In this framework, each data point of the GS model input summarizes ten hours of data. Specifically, the data loader for the GS model generates the input and output of the model as follows: at a given time step, the input includes the actual traffic, weather, and TMI data from the two-hour window before the time step, alongside the weather forecast and scheduled traffic for the next 8 hours starting from the time step. Based on this information, the output of the GS model for each time interval consists of three dimensions. The first dimension represents a binary decision on whether there should be a GS in place for the next hour or not. The second dimension is related to the scope of the GS in the United States, and the third dimension is related to the scope of the GS in Canada (i.e., to determine if the GS impacts airports in Canada).One of the challenges with TMI modeling is the sparsity of TMI events, particularly regarding its scope. To address this challenge in the scope of the GS model output, we implement grouping. The GS scope for the US region is defined based on a list of centers that should be included when the GS is in place. With 20 centers in the US, we utilized historical data to group them into 4 categories. In particular, we summarized our historical data in a graph format where nodes represent centers, and link weights are defined based on the co-occurrence of centers in the scope parameter ofTMIs. By identified strongly connected components in this graph, we were able to partition the centers into four groups. We consider two model structures for the GS Model. Firstly, a hierarchical classification model [10], where the human decision-making for a GS is of hierarchical nature. The decision-maker first decides whether there is a need fora GS, and if the answer is yes, determines the scope. A hierarchical classification model organizes the problem into a class hierarchy, typically a tree or a Directed Acyclic Graph (DAG) structure, and considers the dependency of the decision in the previous step to the next component [10]. Here, we employ the local classifier per level approach, which involves training one multi-class classifier for each level of the class hierarchy. The second structure is the independent structure. In this setting, as the name suggests, we do not consider the dependency of the decisions in the different dimensions of the output of the model. Instead, for each dimension, we train a multi-class classifier independently. Table 1 summarizes GS model statistics for training, validation and testing. The table documents the effect of limiting data to the time steps when there was actually a TMI in place or when a TMI had just terminated. This resulted in a more balanced distribution of the GS class(GS positive class)versus “No GS”(GS negative class), which might help the training process. While JFK and LGA follow very similar distributions, with 40% and 42% GS positive class respectively, EWR has proportionally fewer GS incidents at 28%. Our subsequent phase involves evaluating the performance of both hierarchical structure and independent structure using different state-of-the-art multi-class classifier models such as Random Forest, Decision Trees, K-nearest Neighbors, and Logistic Regression and forecast the duration and scope of the GSs.

Farzan Masrour Shalmani↗

Short-lead seasonal precipitation forecast in northeastern Brazil using an ensemble of artificial neural networks

This study assesses the deterministic and probabilistic forecasting skill of a 1-month-lead ensemble of Artificial Neural Networks (EANN) based on low-frequency climate oscillation indices. The predictand is the February-April (FMA) rainfall in the Brazilian state of Ceará, which is a prominent subject in climate forecasting studies due to its high seasonal predictability. Additionally, the study proposes combining the EANN with dynamical models into a hybrid multi-model ensemble (MME). The forecast verification is carried out through a leave-one-out cross-validation based on 40 years of data. The EANN forecasting skill is compared with traditional statistical models and the dynamical models that compose Ceará’s operational seasonal forecasting system. A spatial comparison showed that the EANN was among the models with the smallest Root Mean Squared Error (RMSE) and Ranked Probability Score (RPS) in most regions. Moreover, the analysis of the area-aggregated reliability showed that the EANN is better calibrated than the individual dynamical models and has better resolution than Multinomial Logistic Regression for above-normal (AN) and below-normal (BN) categories. It is also shown that combining the EANN and dynamical models into a hybrid MME reduces the overconfidence of the extreme categories observed in a dynamically-based MME, improving the reliability of the forecasting system.

54 ENVIRONMENTAL SCIENCES↗

Multivariate Predictive Analytics of Wind Power Data for Robust Control of Energy Storage

Short-term forecasting is frequently identified as an important tool for the effective management of wind generation. However, forecasting errors, inherent to the point forecasts, increase requirements for energy storage and can affect optimal system operation. Probabilistic forecasts can help tackle this issue by providing a proper characterization of forecasting errors in the optimization process. This paper proposes a multivariate model of forecasting data for wind generation. Predictive uncertainty intervals of wind power can be obtained by sampling from the proposed model. The main goal is to use empirical data models without linear or Gaussian approximations of the distributional or temporal variations. The predictive modeling is utilized within a case study of an energy storage system. Finally, a modified robust convex programming is used to maintain the practical robustness and feasibility of the solution based on the sampled scenarios from the model.

17 WIND ENERGY↗

Solar particle event predictions for manned Mars missions

Manned space missions to Mars require consideration of the effects of high radiation doses produced by solar particle events (SPE). Without some provision for protection, the radiation doses from such events can exceed standards for maximum exposure and may be life threatening. Several alternative ways of providing protection require a capability for predicting SPE in time to take some protective actions. The SPE may occur at any time during the eleven year solar cycle so that two year missions cannot be scheduled to insure avoiding them although they are less likely to occur at solar minimum. The present forecasts are sufficiently accurate to use for setting alert modes but are not accurate enough to make yes/no decisions that have major mission operational impacts. Forecasts made for one to two year periods can only be done as probabilistic forecasts where there is a chance of SPE occurring. These are current capabilities but are not likely to change significantly by the year 2000 with the exception of some improvement in the one to ten day forecasts. The effects of SPE are concentrated in solar longitudes near where their parent solar flares occur, which will require a manned Mars mission to carry its own small solar telescope to monitor the development of potentially dangerous solar activity. The preferred telescope complement includes a solar X-ray imager, a hydrogen-alpha scanner, and a solar magnetograph.

Heckman, Gary↗

Effects of Moist Convection on Hurricane Predictability

This study exemplifies inherent uncertainties in deterministic prediction of hurricane formation and intensity. Such uncertainties could ultimately limit the predictability of hurricanes at all time scales. In particular, this study highlights the predictability limit due to the effects on moist convection of initial-condition errors with amplitudes far smaller than those of any observation or analysis system. Not only can small and arguably unobservable differences in the initial conditions result in different routes to tropical cyclogenesis, but they can also determine whether or not a tropical disturbance will significantly develop. The details of how the initial vortex is built can depend on chaotic interactions of mesoscale features, such as cold pools from moist convection, whose timing and placement may significantly vary with minute initial differences. Inherent uncertainties in hurricane forecasts illustrate the need for developing advanced ensemble prediction systems to provide event-dependent probabilistic forecasts and risk assessment.

Zhang, Fuqing↗

An Examination of the Impact of Grid Spacing on WRF Simulations of Wintertime Precipitation in the Mid-Atlantic United States

A large set of deterministic and ensemble forecasts was produced to identify the optimal spacing for forecasting U.S. East Coast snowstorms. WRF forecasts were produced on cloud-allowing (~1-km grid spacing) and convection-allowing (3–4 km) grids, and compared against forecasts with parameterized convection (>~10 km). Performance diagrams were used to evaluate 19 deterministic forecasts from the winter of 2013–14. Ensemble forecasts of five disruptive snowstorms spanning the years 2015–18 were evaluated using various methods to evaluate probabilistic forecasts. While deterministic forecasts using cloud-allowing grids were not better than convection-allowing forecasts, both had lower bias and higher success ratios than forecasts with parameterized convection. All forecasts were underdispersive. Nevertheless, forecasts on the higher-resolution grids were more reliable than those with parameterized convection. Forecasts on the cloud-allowing grid were best able to discriminate areas that received heavy snow and those that did not, while the forecasts with parameterized convection were least able to do so. It is recommended to use convection-resolving and (if computationally possible) to use cloud-allowing forecast grids when predicting East Coast winter storms.

Snowstorms↗

Identification of Robust Terminal-Area Routes in Convective Weather

Convective weather is responsible for large delays and widespread disruptions in the U.S. National Airspace System, especially during summer. Traffic flow management algorithms require reliable forecasts of route blockage to schedule and route traffic. This paper demonstrates how raw convective weather forecasts, which provide deterministic predictions of the vertically integrated liquid (the precipitation content in a column of airspace) can be translated into probabilistic forecasts of whether or not a terminal area route will be blocked. Given a flight route through the terminal area, we apply techniques from machine learning to determine the likelihood that the route will be open in actual weather. The likelihood is then used to optimize terminalarea operations by dynamically moving arrival and departure routes to maximize the expected capacity of the terminal area. Experiments using real weather scenarios on stormy days show that our algorithms recommend that a terminal-area route be modified 30% of the time, opening up 13% more available routes that were forecast to be blocked during these scenarios. The error rate is low, with only 5% of cases corresponding to a modified route being blocked in reality, whereas the original route is in fact open. In addition, for routes predicted to be open with probability 0.95 or greater by our method, 96% of these routes (on average over time horizon) are indeed open in the weather that materializes

Pfeil, Diana Michalek↗

Improving the real-time probabilistic channel flood forecasting by incorporating the uncertainty of inflow using the particle filter

An accurate and reliable real-time flood forecast is crucial for mitigating flood disasters. The errors associated with the inflow boundary forcing data are considered as an important source of uncertainties in hydraulic model. In this paper, a real-time probabilistic channel flood forecasting model is developed with a novel function to incorporate the uncertainty of the forcing inflow. This new approach couples a hydraulic model with the particle filter (PF) data assimilation algorithm, a sequential Bayesian Monte Carlo method. The stage observations at hydrological stations are assimilated at each time step to update the model states in order to improve the next time step’s forecasting. This new approach is tested against a real flood event occurred in the upper Yangtze River. As compared with the open loop simulations, the evaluations of model performance with several deterministic and probabilistic metrics indicate that the accuracy of the ensemble mean prediction and the reliability of the uncertainty quantification are improved pronouncedly as a result of the PF assimilation. Further assessment of the prediction results at different lead times shows that the improvement of model performance deteriorates with the increase of the lead time due to the gradual diminishing of the updating effect for the initial conditions. Based on the analyses of the number of particles and the assimilation frequency, we find that the optimal number of particles can be determined by balancing the model performance and the computation cost, while a high assimilation frequency is preferred to incorporate the emerging observations to update the model states to match the real conditions.

Channel flood forecasting, probabilistic forecast,↗

Use of wind power forecasting in operational decisions.

The rapid expansion of wind power gives rise to a number of challenges for power system operators and electricity market participants. The key operational challenge is to efficiently handle the uncertainty and variability of wind power when balancing supply and demand in ths system. In this report, we analyze how wind power forecasting can serve as an efficient tool toward this end. We discuss the current status of wind power forecasting in U.S. electricity markets and develop several methodologies and modeling tools for the use of wind power forecasting in operational decisions, from the perspectives of the system operator as well as the wind power producer. In particular, we focus on the use of probabilistic forecasts in operational decisions. Driven by increasing prices for fossil fuels and concerns about greenhouse gas (GHG) emissions, wind power, as a renewable and clean source of energy, is rapidly being introduced into the existing electricity supply portfolio in many parts of the world. The U.S. Department of Energy (DOE) has analyzed a scenario in which wind power meets 20% of the U.S. electricity demand by 2030, which means that the U.S. wind power capacity would have to reach more than 300 gigawatts (GW). The European Union is pursuing a target of 20/20/20, which aims to reduce greenhouse gas (GHG) emissions by 20%, increase the amount of renewable energy to 20% of the energy supply, and improve energy efficiency by 20% by 2020 as compared to 1990. Meanwhile, China is the leading country in terms of installed wind capacity, and had 45 GW of installed wind power capacity out of about 200 GW on a global level at the end of 2010. The rapid increase in the penetration of wind power into power systems introduces more variability and uncertainty in the electricity generation portfolio, and these factors are the key challenges when it comes to integrating wind power into the electric power grid. Wind power forecasting (WPF) is an important tool to help efficiently address this challenge, and significant efforts have been invested in developing more accurate wind power forecasts. In this report, we document our work on the use of wind power forecasting in operational decisions.

17 WIND ENERGY↗

VFR Trajectory Forecasting using Deep Generative Model for Autonomous Airspace Operations

To enable the airspace integration of autonomous operations, such as uncrewed aircraft conducting cargo deliveries, there is a need to forecast the positions of the surrounding traffic with which they may interact. This paper focuses on forecasting Visual Flight Rules traffic, a significant source of uncertainty and risk in the airspace, especially around small regional airports, due to the unplanned and often untracked nature of such flights. A deep generative model is developed, trained on historical traffic data at example towered and non-towered airports, and used to predict flight trajectories. Experimental results are presented comparing the performance of variational autoencoder and classical machine learning forecasting when applied to both the towered and non-towered airports over varying time horizons. The results show the advantages of the variational autoencoder in producing accurate probabilistic forecasts over varying time horizons.

uncrewed aircraft↗

VFR Trajectory Forecasting using Deep Generative Model for Autonomous Airspace Operations

To enable the airspace integration of autonomous operations, such as uncrewed aircraft conducting cargo deliveries, there is a need to forecast the positions of the surrounding traffic with which they may interact. This paper focuses on forecasting Visual Flight Rules traffic, a significant source of uncertainty and risk in the airspace, especially around small regional airports, due to the unplanned and often untracked nature of such flights. A deep generative model is developed, trained on historical traffic data at example towered and non-towered airports, and used to predict flight trajectories. Experimental results are presented comparing the performance of variational autoencoder and classical machine learning forecasting when applied to both the towered and non-towered airports over varying time horizons. The results show the advantages of the variational autoencoder in producing accurate probabilistic forecasts over varying time horizons.

uncrewed aircraft↗

Huge ensembles – Part 1: Design of ensemble weather forecasts using spherical Fourier neural operators

Abstract. Simulating low-likelihood high-impact extreme weather events in a warming world is a significant and challenging task for current ensemble forecasting systems. While these systems presently use up to 100 members, larger ensembles could enrich the sampling of internal variability. They may capture the long tails associated with climate hazards better than traditional ensemble sizes. Due to computational constraints, it is infeasible to generate huge ensembles (comprised of 1000–10 000 members) with traditional, physics-based numerical models. In this two-part paper, we replace traditional numerical simulations with machine learning (ML) to generate hindcasts of huge ensembles. In Part 1, we construct an ensemble weather forecasting system based on spherical Fourier neural operators (SFNOs), and we discuss important design decisions for constructing such an ensemble. The ensemble represents model uncertainty through perturbed-parameter techniques, and it represents initial condition uncertainty through bred vectors, which sample the fastest-growing modes of the forecast. Using the European Centre for Medium-Range Weather Forecasts Integrated Forecasting System (IFS) as a baseline, we develop an evaluation pipeline composed of mean, spectral, and extreme diagnostics. With large-scale, distributed SFNOs with 1.1 billion learned parameters, we achieve calibrated probabilistic forecasts. As the trajectories of the individual members diverge, the ML ensemble mean spectra degrade with lead time, consistent with physical expectations. However, the individual ensemble members' spectra stay constant with lead time. Therefore, these members simulate realistic weather states during the rollout, and the ML ensemble passes a crucial spectral test in the literature. The IFS and ML ensembles have similar extreme forecast indices, and we show that the ML extreme weather forecasts are reliable and discriminating. These diagnostics ensure that the ensemble can reliably simulate the time evolution of the atmosphere, including low-likelihood high-impact extremes. In Part 2, we generate a huge ensemble initialized each day in summer 2023, and we characterize the simulations of extremes.

Mahesh, Ankur↗

A copula-based Bayesian method for probabilistic solar power forecasting

With increased penetration of solar energy sources, solar power forecasting has become more crucial and challenging. This paper proposes a copula-based Bayesian approach to improve probabilistic solar power forecasting by capturing the joint distribution between solar power and ambient temperature. A prior forecast distribution is first obtained using different underlying point forecasting models. Parametric and empirical copulas of solar power and temperature are then developed to update the prior distribution to the posterior forecast distribution. A public solar power database is used to demonstrate effectiveness of the proposed method. Numerical results show that the copula-based Bayesian method outperforms the forecasting method that directly uses temperature as a feature. The Bayesian method is also compared with persistent models and show improved performance. Furthermore, this article includes supplementary material (data and code) for reproducibility.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Solar, Wind, and Load Forecasting Dataset for MISO, NYISO, and SPP Balancing Areas

The Performance-based Energy Resource Feedback, Optimization, and Risk Management (PERFORM) program is an initiative intended to foster "a fundamental shift in grid management rooted in an understanding of asset risk and system risk" (ARPA-E 2020). Launched by the Advanced Research Projects Agency-Energy (ARPA-E), the program supports efforts to incorporate uncertainty in electric power decision making. In support of PERFORM, the National Renewable Energy Laboratory (NREL) has produced a set of time-coincident forecasts of solar, wind, and load profiles. As part of Phase I of the PERFORM effort, NREL created a dataset that consists of one year of time-coincident load, wind, and solar actuals and probabilistic forecasts based on data from the Electric Reliability Council of Texas (ERCOT) (Bryce et al. 2023). In Phase II, NREL developed similar datasets for three other U.S. Independent System Operators (ISO): the Midcontinent Independent System Operator (MISO), the New York Independent System Operator (NYISO), and the Southwest Power Pool (SPP).

24 POWER TRANSMISSION AND DISTRIBUTION↗

Evaluation of an open forecasting challenge to assess skill of West Nile virus neuroinvasive disease prediction

Abstract Background West Nile virus (WNV) is the leading cause of mosquito-borne illness in the continental USA. WNV occurrence has high spatiotemporal variation, and current approaches to targeted control of the virus are limited, making forecasting a public health priority. However, little research has been done to compare strengths and weaknesses of WNV disease forecasting approaches on the national scale. We used forecasts submitted to the 2020 WNV Forecasting Challenge, an open challenge organized by the Centers for Disease Control and Prevention, to assess the status of WNV neuroinvasive disease (WNND) prediction and identify avenues for improvement. Methods We performed a multi-model comparative assessment of probabilistic forecasts submitted by 15 teams for annual WNND cases in US counties for 2020 and assessed forecast accuracy, calibration, and discriminatory power. In the evaluation, we included forecasts produced by comparison models of varying complexity as benchmarks of forecast performance. We also used regression analysis to identify modeling approaches and contextual factors that were associated with forecast skill. Results Simple models based on historical WNND cases generally scored better than more complex models and combined higher discriminatory power with better calibration of uncertainty. Forecast skill improved across updated forecast submissions submitted during the 2020 season. Among models using additional data, inclusion of climate or human demographic data was associated with higher skill, while inclusion of mosquito or land use data was associated with lower skill. We also identified population size, extreme minimum winter temperature, and interannual variation in WNND cases as county-level characteristics associated with variation in forecast skill. Conclusions Historical WNND cases were strong predictors of future cases with minimal increase in skill achieved by models that included other factors. Although opportunities might exist to specifically improve predictions for areas with large populations and low or high winter temperatures, areas with high case-count variability are intrinsically more difficult to predict. Also, the prediction of outbreaks, which are outliers relative to typical case numbers, remains difficult. Further improvements to prediction could be obtained with improved calibration of forecast uncertainty and access to real-time data streams (e.g. current weather and preliminary human cases). Graphical Abstract

54 ENVIRONMENTAL SCIENCES↗

Prediction of the Arctic Oscillation in Boreal Winter by Dynamical Seasonal Forecasting Systems

This study assesses the skill of boreal winter Arctic Oscillation (AO) predictions with state-of-the-art dynamical ensemble prediction systems (EPSs): GloSea4, CFSv2, GEOS-5, CanCM3, CanCM4, and CM2.1. Long-term reforecasts with the EPSs are used to evaluate how well they represent the AO and to assess the skill of both deterministic and probabilistic forecasts of the AO. The reforecasts reproduce the observed changes in the large-scale patterns of the Northern Hemispheric surface temperature, upper level wind, and precipitation associated with the different phases of the AO. The results demonstrate that most EPSs improve upon persistence skill scores for lead times up to 2 months in boreal winter, suggesting some potential for skillful prediction of the AO and its associated climate anomalies at seasonal time scales. It is also found that the skill of AO forecasts during the recent period (1997-2010) is higher than that of the earlier period (1983-1996).

Arctic Oscillation↗