Preprints
https://doi.org/10.5194/egusphere-2024-1415
https://doi.org/10.5194/egusphere-2024-1415
22 May 2024
 | 22 May 2024

Assessing the storm surge model performance: What error indicators can measure the skill?

Rodrigo Campos-Caba, Lorenzo Mentaschi, Jacopo Alessandri, Paula Camus, Andrea Mazzino, Franceso Ferrari, Ivan Federico, Michalis Vousdoukas, and Massimo Tondello

Abstract. A well-validated storm surge numerical model is crucial, offering precise coastal hazard information and serving as a basis for extensive databases and advanced data-driven algorithms. However, selecting the best model setup based solely on common error indicators like RMSE or Pearson correlation doesn't always yield optimal results. To illustrate this, we conducted 34-year high-resolution simulations for storm surge under barotropic (BT) and baroclinic (BC) configurations, using atmospheric data from ERA5 and a high-resolution downscaling of the Climate Forecast System Reanalysis (CFSR) developed by the University of Genoa (UniGe). We combined forcing and configurations to produce three datasets: 1) BT-ERA5, 2) BC-ERA5, and 3) BC-UniGe. The model performance was assessed against nearshore station data using various statistical metrics. While RMSE and Pearson correlation suggest BT-ERA5, i.e. the coarsest and simplest setup, as the best model, followed by BC-ERA5, we demonstrate that these indicators aren't always reliable for performance assessment. The most sophisticated model BC-UniGe shows worse values of RMSE or Pearson correlation due to the so-called “double penalty” effect. Here we propose new skill indicators that assess the ability of the model to reproduce the distribution of the observations. This, combined with an analysis of values above the 99th percentile, identifies BC-UniGe as the best model, while ERA5 simulations tend to underestimate the extremes. Although the study focuses on the accurate representation of storm surge by the numerical model, the analysis and proposed metrics can be applied to any problem involving the comparison between time series of simulation and observation.

Publisher's note: Copernicus Publications remains neutral with regard to jurisdictional claims made in the text, published maps, institutional affiliations, or any other geographical representation in this preprint. The responsibility to include appropriate place names lies with the authors.
Rodrigo Campos-Caba, Lorenzo Mentaschi, Jacopo Alessandri, Paula Camus, Andrea Mazzino, Franceso Ferrari, Ivan Federico, Michalis Vousdoukas, and Massimo Tondello

Status: final response (author comments only)

Comment types: AC – author | RC – referee | CC – community | EC – editor | CEC – chief editor | : Report abuse
  • RC1: 'Comment on egusphere-2024-1415', Anonymous Referee #1, 25 Jun 2024
    • AC1: 'Reply on RC1', Rodrigo Campos Caba, 04 Sep 2024
  • RC2: 'Comment on egusphere-2024-1415', Anonymous Referee #2, 28 Jun 2024
    • AC2: 'Reply on RC2', Rodrigo Campos Caba, 04 Sep 2024
  • RC3: 'Comment on egusphere-2024-1415', Anonymous Referee #3, 30 Jun 2024
    • AC3: 'Reply on RC3', Rodrigo Campos Caba, 04 Sep 2024
Rodrigo Campos-Caba, Lorenzo Mentaschi, Jacopo Alessandri, Paula Camus, Andrea Mazzino, Franceso Ferrari, Ivan Federico, Michalis Vousdoukas, and Massimo Tondello
Rodrigo Campos-Caba, Lorenzo Mentaschi, Jacopo Alessandri, Paula Camus, Andrea Mazzino, Franceso Ferrari, Ivan Federico, Michalis Vousdoukas, and Massimo Tondello

Viewed

Total article views: 626 (including HTML, PDF, and XML)
HTML PDF XML Total BibTeX EndNote
348 139 139 626 19 20
  • HTML: 348
  • PDF: 139
  • XML: 139
  • Total: 626
  • BibTeX: 19
  • EndNote: 20
Views and downloads (calculated since 22 May 2024)
Cumulative views and downloads (calculated since 22 May 2024)

Viewed (geographical distribution)

Total article views: 672 (including HTML, PDF, and XML) Thereof 672 with geography defined and 0 with unknown origin.
Country # Views %
  • 1
1
 
 
 
 
Latest update: 15 Oct 2024
Download
Short summary
Development of high-resolution simulations of storm surge in the Northern Adriatic Sea, employing different atmospheric forcing data and physical configurations. Traditional metrics like Pearson correlation and RMSE favor a simulation forced by a coarser database and employing a less sophisticated setup (2D, barotropic). Closer examination allows to identify a baroclinic (3D) model forced by a high-resolution dataset as better able to capture the variability and peak values of the storm surge.