In a warming world of glacier changes, the scientific community has dedicated increasing attention to debris-covered glaciers and their response to climate. A variety of models with distinct complexity and data requirements have been developed and widely used to simulate melt under debris at different sites and scales, but their skills have never been compared. As part of the activities of the International Association of Cryospheric Science (IACS) Debris Covered Glacier Working Group, we present an intercomparison exercise aimed at advancing our understanding of model skills in simulating ice melt under a debris layer. We compare 14 models with different complexity at nine sites in the European Alps, Caucasus, Chilean Andes, Nepalese Himalaya and the Southern Alps of New Zealand, over one melt season. We run the models with measured meteorological data from automatic weather stations and estimated or measured debris properties. We consider four main model categories: i) energy balance models that calculate melt by solving the physics of heat transfer to the debris layer, but require a high amount of input data; ii) a simplified energy balance model; iii) an enhanced temperature-index model; and iv) simple empirical temperature-index models that have been extensively used given their low data requirement but require calibration of their empirical parameters. Model performance is evaluated using on-site measurements of sub-debris melt (for all models) and surface temperature (for models based on the surface energy balance). Our results show that physically-based energy balance models and empirical temperature-index models perform in a distinct manner. At the one end of the spectrum, simple temperature index models are accurate when recalibrated or when using site-specific literature parameters, and show poor results when parameters are uncalibrated. At the other end, energy balance models show a range of performance: the most accurate energy balance models are those with the highest degree of complexity at the atmosphere-debris interface. An important data gap emerged from our experiment: the poor performance of all models at three sites was related to the poor knowledge of debris properties, and specifically of thermal conductivity. Future work should focus on both: i) consistent data acquisition to evaluate existing models and support new model developments; ii) advancing models by accounting for processes such as debris-snow interactions, moisture in the debris and refreezing. We suggest that a systematic effort of model development using a common model framework could be carried out in phase II of the Working Group.