eCite Digital Repository

Using the Schaake shuffle when calibrating ensemble means can be problematic


Shrestha, DL and Robertson, DE and Bennett, JC and Wang, QJ, Using the Schaake shuffle when calibrating ensemble means can be problematic, Journal of Hydrology, 587 Article 124991. ISSN 0022-1694 (2020) [Refereed Article]

Not available

DOI: doi:10.1016/j.jhydrol.2020.124991


The Schaake shuffle is a simple and effective method for re-ordering calibrated ensemble forecasts. It is widely used in forecast calibration methods where realistic spatial and temporal sequences are important. We illustrate a previously unidentified problem with the application of the Schaake shuffle. When the autocorrelation of uncalibrated forecasts is markedly different from observations, the Schaake shuffle cannot guarantee that the calibrated ensemble is reliable when ensemble members are accumulated through time. Accumulations in time and space are particularly important for applications that integrate rainfall over these dimensions, notably hydrological modelling. We demonstrate that ensemble means of uncalibrated forecasts tend to be more autocorrelated than observations. This can cause poor reliability if variables are accumulated across lead times under certain conditions, even if forecasts are perfectly reliable at individual lead times. Specifically, if the following conditions occur, ensemble predictions of accumulated variables tend to be too wide:


Forecasts are more autocorrelated than observations.


Forecasts are skillful; i.e., cross-correlations between forecasts and observations are high.

We show with a real-world case study that these conditions can readily occur. We discuss potential solutions to this issue.

Item Details

Item Type:Refereed Article
Keywords:Schaake shuffle, statistical calibration, poor manís ensemble, ensemble forecasts, autocorrelation, reliability
Research Division:Earth Sciences
Research Group:Hydrology
Research Field:Surface water hydrology
Objective Division:Environmental Policy, Climate Change and Natural Hazards
Objective Group:Understanding climate change
Objective Field:Effects of climate change on Australia (excl. social impacts)
UTAS Author:Bennett, JC (Mr James Bennett)
ID Code:142667
Year Published:2020
Deposited By:Oceans and Cryosphere
Deposited On:2021-02-04
Last Modified:2021-04-28

Repository Staff Only: item control page