Introduction to WCS Sub-R

Statistical Subseasonal Forecasts


Summary


The World Climate Service has released a new statistical long-range forecast product (“Sub-R”) that provides independent guidance for the challenging forecast window between 3 and 6 weeks into the future.  The new product is a direct derivative of the statistical system that produced the first-place temperature forecasts in the 2017-2018 Subseasonal Climate Forecast Rodeo.  Updated daily, and with demonstrated skill at certain times of the year, the new forecasts provide a valuable enhancement to the suite of forecasts and tools available to World Climate Service users.

The WCS Sub-R Interface

The WCS Sub-R Interface



Recommended Use

WCS Sub-R provides completely independent predictions that help adjust and refine expectations for the sub-seasonal forecast period, especially during a window of relatively high skill in May through August.  We have determined that when our calibrated dynamical model forecasts and Sub-R agree during this higher-skill window, then higher confidence can be placed in the dynamical model forecasts.  The reverse is also true: the dynamical models are less reliable when they disagree with the Sub-R forecasts.  We therefore recommend that Sub-R be used as a complement to the independent guidance available from the WCS dynamical multi-model forecasts and WCS subseasonal analog tools.

Origin of Sub-R

In 2017-2018, the World Climate Service (WCS) participated in the US Bureau of Reclamation’s Subseasonal Climate Forecast Rodeo and performed very well, ranking first for all categories combined and second in all but one of the individual forecast categories.  In the temperature categories, the first-place forecasts were submitted by Brian Zimmerman, at the time a graduate student at the University of Wisconsin.  As discussed here, both Brian and the WCS elected to withdraw from the contest and forgo any potential monetary award in order to commercialize the systems that produced the winning forecasts.  Brian is now a team member at Prescient Weather (the creators of WCS) and has implemented a close derivative of his contest-winning solution as a new WCS product. The new forecast product is named WCS Sub-R (“R” stands for Rodeo) and is the first purely statistical, automated forecast to be released as a component of WCS.  While statistical and analog methods have long contributed valuable insight to WCS forecasts, and WCS users are frequent users of analog tools on the WCS website, the Sub-R product represents a step forward in an objective and automated statistical forecasting.

The Model

The statistical system that produces the Sub-R forecasts is a flexible modeling framework that discovers historical relationships between various predictors and subsequent 2m temperature anomalies within a regional forecast domain.  A clustering technique is used to reduce the dimensionality of the problem, and an ensemble of forecasts is generated from a range of model parameters.  The primary inputs to the system are recently observed sea level pressure, sea surface temperature, 2m temperature, and 500mb height, and the recent state of ENSO is used to condition the predictor selection. The Sub-R forecasts are currently available for 2m temperature over three domains: the contiguous US, Europe, and eastern Asia.  Efforts are under way to extend the capability to additional variables and for other domains worldwide.

Forecast Skill

Transparency is one of the core values of the World Climate Service, and so a central theme of the Sub-R development effort has been a detailed investigation of forecast performance.  The skill of the forecasts was examined by running prior forecasts in a purely forward-looking mode from 2010-2018; in other words, the statistical model training was performed over an earlier period (1981-2009), and then forecasts were generated for the verification period (2010-2018).  This method prevents over-fitting and provides a realistic view of forecast skill. The chart below shows the domain-wide anomaly correlation coefficient of the predicted weekly ensemble mean temperature anomalies as a measure of skill over the contiguous US at lead times from three to six weeks.  The skill is quite low overall but shows a clear seasonal variation, with forecasts initialized in May through August having the best performance.  Unfortunately, the current version of the system has no skill in the spring and autumn, and winter-time skill is very marginal.  Further research is ongoing to determine whether a more detailed analysis of the ensemble distribution can provide more useful guidance than the Sub-R ensemble mean.

Subseasonal Forecasts

Monthly Skill Scores of WCS Sub-R Forecasts

To provide easy access to the skill information, the Sub-R product interface includes a toggle to bring up maps of the anomaly correlation for forecasts initialized at the same time of the year as the current forecast selection.  The skill maps are also conditioned on the state of ENSO, because there is a notable dependence on ENSO phase in some regions.  There are often regional variations in skill, and users are advised to consult the skill maps on a regular basis.

Confirmation of Dynamical Models

While the overall skill of the current Sub-R system is quite low, the window of higher skill in May through August suggests that the Sub-R forecasts may have significant value in providing independent confirmation of dynamical model forecasts.  The skill of sub-seasonal dynamical model forecasts is also typically quite low, and therefore an independent source of guidance would be highly valuable. This concept was tested by categorizing the dynamical model forecasts according to whether the Sub-R forecasts agree or disagree in terms of the sign of the predicted temperature anomaly at each location.  The results are unambiguous: during the Sub-R window of higher skill, the dynamical model forecasts are more reliable when Sub-R agrees than when Sub-R disagrees.  In other words, when Sub-R shows a different outcome from the models, the models are less likely to be correct; but when Sub-R agrees, the dynamical model predictions are more trustworthy.

Multi-model ensemble skill scores conditioned on WCS Sub-R forecasts



How to Use Sub-R

The Sub-R forecasts should be regarded as a complement to the WCS calibrated dynamical model forecasts and to other forecast tools such as subseasonal analogs.  Taken in isolation, the Sub-R forecasts have limited skill even within the higher-skill summer window, but as shown above Sub-R can provide a useful independent comparison to the calibrated multi-model dynamical predictions.  We believe the Sub-R prediction system is identifying periods of time when the atmosphere is more predictable than at other times. Our recommendation is that the Sub-R forecasts be monitored carefully during the higher-skill summer period and consulted regularly at other times of the year.  While the historical skill analysis does not reveal useful skill outside of summer, it is still possible that high-amplitude Sub-R signals can be used as independent context to judge whether the dynamical model forecasts are plausible.  Additional WCS research is ongoing to determine how best to use Sub-R at different times of the year, and of course we also seek to improve the fundamental skill of the system by including additional predictors and exploring different statistical modeling and machine learning possibilities.

Next Step

We are offering free trials of the World Climate Service Sub-R capability during the summer of 2019.   Please use the form in the upper right corner of this page to request a trial.