Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Forecast Comparison Based on Random Walks

Forecast Comparison Based on Random Walks This paper proposes a procedure based on random walks for testing and visualizing differences in forecast skill. The test is formally equivalent to the sign test and has numerous attractive statistical properties, including being independent of distributional assumptions about the forecast errors and being applicable to a wide class of measures of forecast quality. While the test is best suited for independent outcomes, it provides useful information even when serial correlation exists. The procedure is applied to deterministic ENSO forecasts from the North American Multimodel Ensemble and yields several revealing results, including 1) the Canadian models are the most skillful dynamical models, even when compared to the multimodel mean; 2) a regression model is significantly more skillful than all but one dynamical model (to which it is equally skillful); and 3) in some cases, there are significant differences in skill between ensemble members from the same model, potentially reflecting differences in initialization. The method requires only a few years of data to detect significant differences in the skill of models with known errors/biases, suggesting that the procedure may be useful for model development and monitoring of real-time forecasts. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Monthly Weather Review American Meteorological Society

Forecast Comparison Based on Random Walks

DelSole, Timothy; Tippett, Michael K.
Monthly Weather Review , Volume 144 (2) – Jun 8, 2015
Download PDF
12 pages

Loading next page...
 
/lp/american-meteorological-society/forecast-comparison-based-on-random-walks-QmSwDPz8bP

References (10)

Publisher
American Meteorological Society
Copyright
Copyright © 2015 American Meteorological Society
ISSN
0027-0644
eISSN
1520-0493
DOI
10.1175/MWR-D-15-0218.1
Publisher site
See Article on Publisher Site

Abstract

This paper proposes a procedure based on random walks for testing and visualizing differences in forecast skill. The test is formally equivalent to the sign test and has numerous attractive statistical properties, including being independent of distributional assumptions about the forecast errors and being applicable to a wide class of measures of forecast quality. While the test is best suited for independent outcomes, it provides useful information even when serial correlation exists. The procedure is applied to deterministic ENSO forecasts from the North American Multimodel Ensemble and yields several revealing results, including 1) the Canadian models are the most skillful dynamical models, even when compared to the multimodel mean; 2) a regression model is significantly more skillful than all but one dynamical model (to which it is equally skillful); and 3) in some cases, there are significant differences in skill between ensemble members from the same model, potentially reflecting differences in initialization. The method requires only a few years of data to detect significant differences in the skill of models with known errors/biases, suggesting that the procedure may be useful for model development and monitoring of real-time forecasts.

Journal

Monthly Weather Review – American Meteorological Society

Published: Jun 8, 2015

There are no references for this article.