On combining climate models into weighted ensembles
Abstract. Several methods have been proposed and used to refine estimates of future climate change based on combined output from comprehensive climate models. While previously the so-called model democracy approach was used to combine model predictions, where every model is given equal weight, it is now widely accepted that using model weights that account for model performance and model independence is necessary to obtain more reliable results. However, most existing approaches rely, implicitly or explicitly, on a similar statistical basis, while describing things in different ways. Here we distinguish between approaches that are based on the performance of individual models (individual performance weighting) and approaches that are based on the performance of the weighted ensemble as a whole (ensemble performance weighting). At the same time, we formulate both in probabilistic Bayesian terms to make their application and comparison straightforward. Using simple constructed examples, we demonstrate that the ensemble performance weighting approach implicitly accounts for co-dependencies among models, which arguably makes the computation of independence weights for the purpose of model weighting obsolete. We also show that a set of weighted models within the ensemble weighting approach will naturally tend to artificially reduce uncertainty and that this is strongly influenced by the choice of the prior distribution over weight vectors. The distinction between individual and ensemble performance weighting is both methodological and conceptual. Formulating both approaches in general probabilistic Bayesian terms as done here, can serve as a common basis for future developments with regard to ensemble model weighting in Earth system science.
The manuscript "On combining climate models into weighted ensembles" addresses model ensembles with specific weights per model based on two approaches: individual performance weighting and ensemble performance weighting. It is argued that, as regarding ensemble performance weghting weights of individual models depend on the tuning of the whole ensemble, co-dependencies of the models are implicitely accounted for. The intention of the manuscript is to delineate the two concepts based on Bayesian probabilites particularly regarding considering inter-model dependencies. Also different methodologies inside the two approaches are analysed.
In general, the manuscript is well written and comprehensible. The purpose of the study is met. Nevertheless, it would be beneficial for modelers and other people dealing with climate model data but not proficient in Bayesian statistics, to give a brief introduction regarding the most important Bayesian concepts used inside the manuscript (prior, posterior, MCMC, Dirichlet distributions, ...).
I recommend to publish the article on condition of adding the aforementioned explications as well as implementation of the following minor revisions:
Line 67, 88, Box 1, 95, 100, 101, 110, 117, 184, ... [general]: I would recommend to refrain from the term "prediction" when talking about climate models
Box 1: it would be good, to include something like i=[1,N] or i=1,..,N in Box 1; in Equation 4 N is used but not defined, the first explanation of N is in line 136 (although it is quite intuitive what N should mean)
equation 5: superscript v missing for y in P(y|...)
Lines 114–120: please add reference(s) for this paragraph
L140: please add more information and reference reagrding Dirichlet distribution (if not done in the general explanations regarding Bayesian statistics)
Figure 1: a) although clear from the text, please add information that model 1 and 2 are hidden below model 1 in the two lower rows
Line 194: please inform the reader that i.i.d. = independent and identically distributed
Lines 217/218: please include reference for "common pri distribution ... Invserse-gamma"
Line 266: Reference for ERA5 missing (question mark in brackets)
Line 278: This (always same weight for M1 irrspective of number of M2 copies) only holds true for ensemble performance weighting (not for individual performance weighting) [as also can be seen in Figure 6]
Figure 7: is each point in Fig. 7a a different location and Fig. 7b the spatial variance? From equation 8 and the discussion I deduce that this is not the case. So I don't understand the single points. In Figure 6 there is just one weight for each model and now we have plenty of them? Or is this the probability distribution of the weights again?
Line 332: please write out in full SSP5-8.5
Line 333: please use (en) dash instead of hyphen for time span
Figure 8: Please add unit to x-axis. Is this K per W/m² or K per doubling of CO2?
Figure 9: "Projected" instead of "Predicted" please, pleasse write out in full SSP5-8.5
Line 351: I cannot find any "thin dashed green lines in Fig. 9"
Line 368: "ECS as performance metric" this is no metric but a variable considered for weighting (which is then applied to temperature data) -> a basis for assessing model performance
Lines 393–396: This is a good point. The other way round, also holds true: models with very similar model structures can lead to different projections, e.g. if they are based on the same ocean model but the atmosphere is different, ...
Please check: sometimes, blank spaces between commas ("e.g.,see") or dots ("vs.0.5820") and the following words/numbers are missing