DELWAVE 1.0: Deep-learning surrogate model of surface wave climate in the Adriatic Basin

Mlakar, Peter; Ricchi, Antonio; Carniel, Sandro; Bonaldo, Davide; Ličer, Matjaž

doi:https://doi.org/10.5194/egusphere-2023-718

Preprints

https://doi.org/10.5194/egusphere-2023-718

Preprints

01 Jun 2023

| 01 Jun 2023

DELWAVE 1.0: Deep-learning surrogate model of surface wave climate in the Adriatic Basin

Peter Mlakar, Antonio Ricchi, Sandro Carniel, Davide Bonaldo, and Matjaž Ličer

Abstract. We propose a new point-prediction DEep Learning WAVe Emulating model (DELWAVE) which successfully emulates the behaviour of a numerical surface ocean wave model (SWAN) at a sparse set of locations, thus enabling numerically cheap large-ensemble prediction over synoptic to climate timescales. DELWAVE training inputs consist of 6-hourly surface COSMO-CLM wind fields during period 1971–1998, while its targets are significant wave height, mean wave period and mean wave direction. Testing input set consists of surface winds during 1998–2000 and cross-validation period is the far-future climate timewindow of 2071–2100. DELWAVE is constructed to have a convolution-based atmospheric encoder block, followed by a temporal collapse block and finally a regression block. Random importance-sampling was performed to better model underpopulated tails of variable data distributions. Detailed ablation studies were performed to determine optimal performance regarding input fields, temporal horizon of the training set and network architecture. DELWAVE reproduces SWAN model significant wave heights with a mean absolute error (MAE) between 5 and 10 cm, mean wave directions with a MAE of 10°–25° and mean wave period with a MAE of 0.2 s. DELWAVE is able to accurately emulate multi-modal mean wave direction distributions, related to dominant wind regimes in the basin. We use wave power analysis from linearized wave theory to explain prediction errors in the long-period limit during southeasterly conditions, indicating, as expected, that non-local generation of swell poses a more difficult challenge during long-fetched Scirocco than during cross-basin Bora flow. We present a storm analysis of DELWAVE, employing threshold-based metrics of precision and recall to show that DELWAVE reaches a very high score (both metrics over 95 %) of storm detection. SWAN and DELWAVE time series are compared against each other in the end-of-century scenario (2071–2100), and compared to the control conditions in the 1971–2000 period. Good agreement between DELWAVE and SWAN is found when considering climatological statistics, with a small (≤5 %), though systematic, underestimate of 99th percentile values. Compared to control climatology over all wind directions, the mismatch between DELWAVE and SWAN is generally small compared to the difference between scenario and control conditions, suggesting that the noise introduced by surrogate modeling is substantially weaker than the climate change signal.

Received: 17 Apr 2023 – Discussion started: 01 Jun 2023

Publisher's note: Copernicus Publications remains neutral with regard to jurisdictional claims made in the text, published maps, institutional affiliations, or any other geographical representation in this preprint. The responsibility to include appropriate place names lies with the authors.

Download & links

Preprint (PDF, 8785 KB)

Notice on discussion status
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
Preprint (8785 KB)

Download & links

The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.

Journal article(s) based on this preprint

17 Jun 2024

DELWAVE 1.0: deep learning surrogate model of surface wave climate in the Adriatic Basin

Peter Mlakar, Antonio Ricchi, Sandro Carniel, Davide Bonaldo, and Matjaž Ličer

Geosci. Model Dev., 17, 4705–4725, https://doi.org/10.5194/gmd-17-4705-2024,https://doi.org/10.5194/gmd-17-4705-2024, 2024

Short summary

Peter Mlakar, Antonio Ricchi, Sandro Carniel, Davide Bonaldo, and Matjaž Ličer

Interactive discussion

Status: closed

CC1:
'Comment on egusphere-2023-718', Haoyu Jiang, 17 Jul 2023

I came across this article by accident and gave it a quick skim.

In such a small, almost closed sea basin, where there are no long-propagating swells, predicting waves with deep learning is very simple. The prediction of single-point wave integral parameters (as well as wave spectra) can be achieved with a very simple CNN, and the results are not worse than those reported in the paper.

However, switching to a larger sea basin increases the number of input parameters dramatically, making it extremely difficult to train the model. Besides, training one model per point to achieve a numerical wave pattern surrogate even more leads to a very large model size.

Citation: https://doi.org/10.5194/egusphere-2023-718-CC1
- CC2: 'Reply on CC1', Haoyu Jiang, 17 Jul 2023
  
  Sorry, I posted this in the wrong place.
  
  Please ignore this comment.
  It will be nice if the editor can remove this comment.
  
  Citation: https://doi.org/10.5194/egusphere-2023-718-CC2
RC1: 'Comment on egusphere-2023-718', Giacomo Capodaglio, 09 Oct 2023

This is an interesting paper and the results are very promising. There are a few things that need improvement though, for instance parts of the manuscript should have more mathematical rigor instead of attempting to describe in words certain formulations. A more detailed description on computational times should also be included, since the focus is almost entirely on the comparison between numerical predictions of SWAN and DELWAVE. This is certainly important, but your initial argument was that DELWAVE was a mean to save computational time, and I did not find a convincing argument as to whether you have shown this in your paper. Please find in the attached document a complete list of minor and major points that I would like the authors to address before I can reconsider this manuscript for publication.

Citation: https://doi.org/10.5194/egusphere-2023-718-RC1
RC2: 'Comment on egusphere-2023-718', Anonymous Referee #2, 06 Nov 2023

This study presents an emulator for wind-waves in the Adriatic Sea that is claimed to be capable to produce “numerically cheap large-ensemble predictions over synoptic to climate time scales”. While the procedure following which the emulator was built, trained and tested is well-described and seems sound (this should be evaluated by an expert in machine learning techniques), the setup and evaluation of the SWAN wave model is lacking. Further, no discussion about the choices made to build the emulator is present. Consequently, although the scientific significance and quality of this manuscript for the understanding of future wind-wave climate in the Adriatic Sea has the potential to be high, it is, in the current stage, not convincing. I list below some major comments to support the authors with the resubmission of their manuscript.
Major comments:
The main issue with the article is the relatively low resolution used in the COSMO-CLM and SWAN models. In particular, the SWAN model horizontal resolutions reach at best 2 km for the AA station but up to 8-9 km for the MB station and about 6 km for the remaining stations. In the southern Adriatic the resolutions are close to the Med-CORDEX regional climate models covering the full Mediterranean Sea at about 12 km of resolution. Further, from Bonaldo et al. (2020), it seems that “the minimum water depth in the model grid equals approximately 8 m”. In my opinion this defy the purpose of using an emulator for the wind-waves at only 6 locations of interest. Such an approach should, in fact, allow to reach a resolution of few (maybe hundred) meters at locations of interest (where bathymetry should be updated with observations; e.g., multi-beam or LiDAR) in order to properly resolve refraction, diffraction, shoaling, reflection, etc. of the waves along the coastline.
The choice of the locations where the emulators were built is also a bit puzzling and unexplained. Why the stations OB2 and OB3 are of any interest? Why the emulator results are only presented for AA, OB and MB and not at Grado, for example? Why emulators were not built for all the major coastal cities along the Adriatic coast and/or the Adriatic LNG terminal, the major commercial harbors like Koper, etc.? I understand the choice to include the wave buoy stations but not to limit the emulators to it.
This leads to another important point. The evaluation of the SWAN model against observations is not presented. The COSMO-CLM model has been evaluated for the (EURO-?) CORDEX domain by forcing its boundaries with reanalysis (i.e., ERA-Interim). The evaluation of the SWAN model should thus be performed during the period of this control run for extreme events (and not as a climatology like done in Bonaldo et al., 2020) and compare with the available observations in order to assess the capacity of the COSMO-CLM and SWAN models to reproduce bora/sirocco winds and wave parameters, respectively. Without such an evaluation for extreme events, the skills of the COSMO-CLM and SWAN models during sirocco/bora events, and, hence, of the emulator, cannot be thoroughly assessed and no conclusion about the quality of the results presented in the manuscript can be reached (i.e., an emulator can only be as good as the geoscientific models it is built with).
In terms of the technical implementation of the emulator, I would recommend the article to be reviewed by an expert in machine learning. The article is presenting a lot of details about the way the emulator was built that only such a specialist can accurately review.
Another important question not discussed in this paper is whether or not the emulator can be used with other regional atmospheric forcing than COSMO-CLM. In terms of producing robust ensembles to cover the climate uncertainty, it is crucial to use as many different atmospheric climate models as possible. As nevertheless the COSMO-CLM model is corrected with the ERA5 reanalysis, one can even ask, why not using directly the CMIP6 100-km resolution ensemble of atmospheric models (obviously corrected with ERA5 in order to catch extreme events) to implement, test and train the emulator. As it seems that the authors limit their study to wave buoy locations, it could even be envisioned to directly use the wave observations to build such an emulator (and maybe even skip the SWAN modelling). I am not suggesting that it is a better method but I am just highlighting that, over all, there is a lack of discussion concerning both the choices made to build the emulator and the practical use of the emulator for climate studies.
Finally, the last major point is that the authors did not prove that their emulator was cheaper than, for example, look-up tables that are commonly used for wind-waves. In combination with the relatively low resolution of the SWAN model and the lack of evaluation of the COSMO-CLM and SWAN models during storm events, the manuscript thus fails to prove the added value of the DELWAVE model.
Overall, even if the mathematical exercise of setting up an emulator for wind-waves in the Adriatic Sea is interesting as such, I am not convinced that, the DELWAVE emulator can really be used for the intended purposes stated in the introduction: study “morphodynamic processes”, the “safety and durability of human infrastructures, along the coast and offshore” and assess “the feasibility and improving the design of wave energy converter facilities”.
Reference:
Bonaldo, D., Bucchignani, E., Pomaro, A., Ricchi, A., Sclavo, M., and Carniel, S.: Wind waves in the Adriatic Sea under a severe climate change scenario and implications for the coasts, International Journal of Climatology, 40, 5389–5406, https://doi.org/10.1002/joc.6524, 2020.

Citation: https://doi.org/10.5194/egusphere-2023-718-RC2
AC1: 'Comment on egusphere-2023-718', Peter Mlakar, 01 Dec 2023

We would like to thank all reviewers and the community for their invaluable feedback.
We provide our responses in the attachment included herein.

Citation: https://doi.org/10.5194/egusphere-2023-718-AC1

Interactive discussion

Status: closed

CC1:
'Comment on egusphere-2023-718', Haoyu Jiang, 17 Jul 2023

I came across this article by accident and gave it a quick skim.

In such a small, almost closed sea basin, where there are no long-propagating swells, predicting waves with deep learning is very simple. The prediction of single-point wave integral parameters (as well as wave spectra) can be achieved with a very simple CNN, and the results are not worse than those reported in the paper.

However, switching to a larger sea basin increases the number of input parameters dramatically, making it extremely difficult to train the model. Besides, training one model per point to achieve a numerical wave pattern surrogate even more leads to a very large model size.

Citation: https://doi.org/10.5194/egusphere-2023-718-CC1
- CC2: 'Reply on CC1', Haoyu Jiang, 17 Jul 2023
  
  Sorry, I posted this in the wrong place.
  
  Please ignore this comment.
  It will be nice if the editor can remove this comment.
  
  Citation: https://doi.org/10.5194/egusphere-2023-718-CC2
RC1: 'Comment on egusphere-2023-718', Giacomo Capodaglio, 09 Oct 2023

This is an interesting paper and the results are very promising. There are a few things that need improvement though, for instance parts of the manuscript should have more mathematical rigor instead of attempting to describe in words certain formulations. A more detailed description on computational times should also be included, since the focus is almost entirely on the comparison between numerical predictions of SWAN and DELWAVE. This is certainly important, but your initial argument was that DELWAVE was a mean to save computational time, and I did not find a convincing argument as to whether you have shown this in your paper. Please find in the attached document a complete list of minor and major points that I would like the authors to address before I can reconsider this manuscript for publication.

Citation: https://doi.org/10.5194/egusphere-2023-718-RC1
RC2: 'Comment on egusphere-2023-718', Anonymous Referee #2, 06 Nov 2023

This study presents an emulator for wind-waves in the Adriatic Sea that is claimed to be capable to produce “numerically cheap large-ensemble predictions over synoptic to climate time scales”. While the procedure following which the emulator was built, trained and tested is well-described and seems sound (this should be evaluated by an expert in machine learning techniques), the setup and evaluation of the SWAN wave model is lacking. Further, no discussion about the choices made to build the emulator is present. Consequently, although the scientific significance and quality of this manuscript for the understanding of future wind-wave climate in the Adriatic Sea has the potential to be high, it is, in the current stage, not convincing. I list below some major comments to support the authors with the resubmission of their manuscript.
Major comments:
The main issue with the article is the relatively low resolution used in the COSMO-CLM and SWAN models. In particular, the SWAN model horizontal resolutions reach at best 2 km for the AA station but up to 8-9 km for the MB station and about 6 km for the remaining stations. In the southern Adriatic the resolutions are close to the Med-CORDEX regional climate models covering the full Mediterranean Sea at about 12 km of resolution. Further, from Bonaldo et al. (2020), it seems that “the minimum water depth in the model grid equals approximately 8 m”. In my opinion this defy the purpose of using an emulator for the wind-waves at only 6 locations of interest. Such an approach should, in fact, allow to reach a resolution of few (maybe hundred) meters at locations of interest (where bathymetry should be updated with observations; e.g., multi-beam or LiDAR) in order to properly resolve refraction, diffraction, shoaling, reflection, etc. of the waves along the coastline.
The choice of the locations where the emulators were built is also a bit puzzling and unexplained. Why the stations OB2 and OB3 are of any interest? Why the emulator results are only presented for AA, OB and MB and not at Grado, for example? Why emulators were not built for all the major coastal cities along the Adriatic coast and/or the Adriatic LNG terminal, the major commercial harbors like Koper, etc.? I understand the choice to include the wave buoy stations but not to limit the emulators to it.
This leads to another important point. The evaluation of the SWAN model against observations is not presented. The COSMO-CLM model has been evaluated for the (EURO-?) CORDEX domain by forcing its boundaries with reanalysis (i.e., ERA-Interim). The evaluation of the SWAN model should thus be performed during the period of this control run for extreme events (and not as a climatology like done in Bonaldo et al., 2020) and compare with the available observations in order to assess the capacity of the COSMO-CLM and SWAN models to reproduce bora/sirocco winds and wave parameters, respectively. Without such an evaluation for extreme events, the skills of the COSMO-CLM and SWAN models during sirocco/bora events, and, hence, of the emulator, cannot be thoroughly assessed and no conclusion about the quality of the results presented in the manuscript can be reached (i.e., an emulator can only be as good as the geoscientific models it is built with).
In terms of the technical implementation of the emulator, I would recommend the article to be reviewed by an expert in machine learning. The article is presenting a lot of details about the way the emulator was built that only such a specialist can accurately review.
Another important question not discussed in this paper is whether or not the emulator can be used with other regional atmospheric forcing than COSMO-CLM. In terms of producing robust ensembles to cover the climate uncertainty, it is crucial to use as many different atmospheric climate models as possible. As nevertheless the COSMO-CLM model is corrected with the ERA5 reanalysis, one can even ask, why not using directly the CMIP6 100-km resolution ensemble of atmospheric models (obviously corrected with ERA5 in order to catch extreme events) to implement, test and train the emulator. As it seems that the authors limit their study to wave buoy locations, it could even be envisioned to directly use the wave observations to build such an emulator (and maybe even skip the SWAN modelling). I am not suggesting that it is a better method but I am just highlighting that, over all, there is a lack of discussion concerning both the choices made to build the emulator and the practical use of the emulator for climate studies.
Finally, the last major point is that the authors did not prove that their emulator was cheaper than, for example, look-up tables that are commonly used for wind-waves. In combination with the relatively low resolution of the SWAN model and the lack of evaluation of the COSMO-CLM and SWAN models during storm events, the manuscript thus fails to prove the added value of the DELWAVE model.
Overall, even if the mathematical exercise of setting up an emulator for wind-waves in the Adriatic Sea is interesting as such, I am not convinced that, the DELWAVE emulator can really be used for the intended purposes stated in the introduction: study “morphodynamic processes”, the “safety and durability of human infrastructures, along the coast and offshore” and assess “the feasibility and improving the design of wave energy converter facilities”.
Reference:
Bonaldo, D., Bucchignani, E., Pomaro, A., Ricchi, A., Sclavo, M., and Carniel, S.: Wind waves in the Adriatic Sea under a severe climate change scenario and implications for the coasts, International Journal of Climatology, 40, 5389–5406, https://doi.org/10.1002/joc.6524, 2020.

Citation: https://doi.org/10.5194/egusphere-2023-718-RC2
AC1: 'Comment on egusphere-2023-718', Peter Mlakar, 01 Dec 2023

We would like to thank all reviewers and the community for their invaluable feedback.
We provide our responses in the attachment included herein.

Citation: https://doi.org/10.5194/egusphere-2023-718-AC1

Peer review completion

AR: Author's response | RR: Referee report | ED: Editor decision | EF: Editorial file upload

AR by Peter Mlakar on behalf of the Authors (02 Dec 2023) Author's response Author's tracked changes Manuscript

ED: Referee Nomination & Report Request started (08 Dec 2023) by Simone Marras

RR by Anonymous Referee #1 (24 Dec 2023)

RR by Anonymous Referee #3 (01 Mar 2024)

RR by Anonymous Referee #4 (03 Apr 2024)

Suggestions for revision or reasons for rejection

This paper went to a throughout first revision and the authors addressed the comments raised be the reviewers with a detailed response and clarified the manuscript accordingly.

It is particular interesting that the authors have tested the model trained on current period (1971-2000) to future climate 2071-2100 and benchmarked their model against the SWAN model. It is a common critique to machine learning model that their application is doubtful for different climates.
This paper is the first paper that I am aware of, that address this question in a direct manner by comparing the SWAN and a deep learning model (DELWAVE) in this context which I consider as a difficult test. It is encouraging to see that DELWAVE performs quite well in this case (Figure 10).

Many comments of the reviewer 1 were related to the clarification of some technical concepts in machine learning. The authors have addressed these concerns and as a result the revised manuscript is now more approachable for the readers of GMD.

Questions of reviewers 2 were more focused on the wave modelling aspect which have also been taking into account in the revised manuscript.

While reading the manuscript I have no comments that would require changes to the manuscript but I have two suggestions that I might be interesting for future studies.

It is quite interesting that training only needs wave time series at six different location. This requires that the neural networks emulates all processes between the location and has an implicit latent representation of them. This contrast with other application where the emulator uses a regular grid of the full domain (e.g. GraphCast, 10.1126/science.adi2336). It might be interesting to compare both approach in the future. While in the current approach, the neural network can be trained only from observational data, the approach in GraphCast would require model result (typically with data assimilation) over the whole domain.

Equation 7: grid encoding matrix: it is easy for the network to see that (i,j) and (i,j+1) are direct neighbors (jump of 1/(nx ny) , but the jump of the encoding between (i,j) and (i+1,j) is larger (1/nx). Maybe in future work the authors can also compare their approach to use direct the indices i and j (scaled linearly between -0.5 and 0.5). The grid encoding would thus have two elements instead of one. Potentially, this can reduce a bit the training time, but I do not think that this will have an impact on the results.

Hide

ED: Reject (01 Mar 2024) by Simone Marras

ED: Publish as is (11 Apr 2024) by Simone Marras

AR by Peter Mlakar on behalf of the Authors (22 Apr 2024) Manuscript

Journal article(s) based on this preprint

17 Jun 2024

DELWAVE 1.0: deep learning surrogate model of surface wave climate in the Adriatic Basin

Peter Mlakar, Antonio Ricchi, Sandro Carniel, Davide Bonaldo, and Matjaž Ličer

Geosci. Model Dev., 17, 4705–4725, https://doi.org/10.5194/gmd-17-4705-2024,https://doi.org/10.5194/gmd-17-4705-2024, 2024

Short summary

Peter Mlakar, Antonio Ricchi, Sandro Carniel, Davide Bonaldo, and Matjaž Ličer

Viewed

Total article views: 583 (including HTML, PDF, and XML)

HTML	PDF	XML	Total	BibTeX	EndNote
393	154	36	583	17	23

HTML: 393
PDF: 154
XML: 36
Total: 583
BibTeX: 17
EndNote: 23

Views and downloads (calculated since 01 Jun 2023)

Month	HTML	PDF	XML	Total
Jun 2023	81	28	7	116
Jul 2023	44	16	6	66
Aug 2023	33	4	1	38
Sep 2023	36	10	1	47
Oct 2023	57	21	5	83
Nov 2023	43	16	6	65
Dec 2023	18	8	2	28
Jan 2024	13	11	1	25
Feb 2024	14	5	2	21
Mar 2024	23	13	0	36
Apr 2024	14	7	3	24
May 2024	15	11	1	27
Jun 2024	2	4	1	7

Cumulative views and downloads (calculated since 01 Jun 2023)

Month	HTML	PDF	XML	Total
Jun 2023	81	28	7	116
Jul 2023	44	16	6	66
Aug 2023	33	4	1	38
Sep 2023	36	10	1	47
Oct 2023	57	21	5	83
Nov 2023	43	16	6	65
Dec 2023	18	8	2	28
Jan 2024	13	11	1	25
Feb 2024	14	5	2	21
Mar 2024	23	13	0	36
Apr 2024	14	7	3	24
May 2024	15	11	1	27
Jun 2024	2	4	1	7

Viewed (geographical distribution)

Total article views: 564 (including HTML, PDF, and XML) Thereof 564 with geography defined and 0 with unknown origin.

Country	#	Views	%

Latest update: 17 Jun 2024

Download

The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.

Preprint (8785 KB)
Metadata XML

Short summary

We propose a new point-prediction DEep Learning WAVe Emulating model (DELWAVE) which successfully emulates the ocean wave model (SWAN) over synoptic to climate timescales. Compared to control climatology over all wind directions, the mismatch between DELWAVE and SWAN is generally small compared to the difference between scenario and control conditions, suggesting that the noise introduced by surrogate modeling is substantially weaker than the climate change signal.


Total:	0
HTML:	0
PDF:	0
XML:	0