Making Sense of Long Range Forecasts

In an October 31 blog post, distinguished meteorologist Cliff Mass of the University of Washington commented on the degree of usefulness of long range forecasts, with the title, “Extended Forecasts Are Not Reliable“.

Dr. Mass highlights the lack of deterministic skill of temperature and precipitation forecasts in the dynamical forecast models beyond two weeks and illustrates the problem of “poor forecasts” with the October 2019 U.S. temperature forecast from the Climate Prediction Center, which he calls “a big miss”.

Long range forecasts: correct interpretation required

While some of the points made in the post are indisputably correct, and the main message probably resonates with the general public, we humbly suggest that the discussion misconstrues the value of long range forecasts and perpetuates certain misunderstandings about the very nature of extended-range forecasting. A couple of specific points can be made.

First, we should note that credible long-lead forecasts are presented in terms of probability and confidence, rather than in black-and-white deterministic terms. For example, the October 2019 CPC forecast was a probability forecast; it called for a 50-60% probability of above-normal (upper tercile) temperatures over the southwestern U.S., and lower (but still enhanced) probabilities of unusual warmth elsewhere in the lower 48. Implied in the forecast was a reduced, but certainly not zero, probability of near-normal or below-normal temperatures. It is therefore incorrect to say that the forecast called for “MUCH above normal [conditions] over the southwest U.S.”, because the forecast neither made a categorical prediction nor included any information about the magnitude of temperature departures from normal.

Example Climate Prediction Center forecast

The distinction between a probability forecast and a categorical statement of the expected outcome may seem like a technicality or even an excuse for the forecaster who gets it “wrong”, but in our view, it’s critical to understand the difference. The fact of the matter is that long-range forecasting is difficult, and most of the time there are few strong indications of likely departures from normal, whether from the dynamical models or other guidance. Therefore the only reasonable way to approach the problem is to present the forecast in terms of probabilities; and the performance of the forecast must be judged in the same framework, rather than on the basis of one (or just a few) perceived successes or failures.

Notice that we said “most of the time” the forecast is relatively uncertain. There are also times when important signals emerge and confidence is much higher than normal; these are occasions when the long-lead guidance tends to agree, the signals are physically consistent, and the probability of significant anomalies is relatively high. If the forecasts are presented in terms of probabilities, and if those probabilities are well-founded (i.e. based on objective scientific methods), then on these occasions users can confidently take action and reap the benefits of success. The World Climate Service forecasts for the European summers of 2018 and 2019 were examples of high-confidence forecasts that proved successful.

A second point to make in connection with Dr. Mass’ blog post is that there is much more to long-lead forecasting than simply following the dynamical model predictions. Dr. Mass is correct, of course, that the models lose the ability to anticipate daily weather patterns after two weeks (although there is a lot of useful skill at far longer leads in other aspects of the climate), but long-range forecasts also depend on statistical and historical analog methods. These independent tools, which rely on systematic relationships between different components of the ocean/atmosphere system, often prove just as useful – and sometimes even more so – than the dynamical models. The World Climate Service provides access to a wealth of tools and information that complements the dynamical model guidance, and the experienced forecaster is able to create a more skillful and valuable forecast by using the full range of forecast guidance, both dynamical and statistical.

To sum up, we believe that long range forecasts, when prepared and communicated properly, are more skillful, credible, and valuable than Dr. Mass suggests in his recent blog post. To be sure, enormous challenges remain, as skill is limited and high-confidence forecasts are relatively rare; and moreover the conceptual basis of (probabilistic) long lead forecasts remains difficult to communicate and is often hard for users to accept. Nevertheless, with more than a decade of experience in this sector, the World Climate Service is well-placed to solve these challenges and expects to remain at the forefront of real-world applications of valuable long-lead guidance for the next decade and beyond.