HIDRA3: a robust deep-learning model for multi-point ensemble sea level forecasting

Rus, Marko; Mihanović, Hrvoje; Ličer, Matjaž; Kristan, Matej

doi:https://doi.org/10.5194/egusphere-2024-2068

Preprints

https://doi.org/10.5194/egusphere-2024-2068

Preprints

29 Aug 2024

| 29 Aug 2024

HIDRA3: a robust deep-learning model for multi-point ensemble sea level forecasting

Marko Rus, Hrvoje Mihanović, Matjaž Ličer, and Matej Kristan

Abstract. Accurate modeling of sea level and storm surge dynamics with several day-long temporal horizons is essential for effective coastal flood response and the protection of coastal communities and economies. The classical approach to this challenge involves computationally intensive ocean models that typically calculate sea levels relative to the geoid, which must then be correlated with local tide gauge observations of sea surface heights (SSH). A recently proposed deep learning model, HIDRA2, avoids numerical simulations while delivering competitive forecasts. Its forecast accuracy depends on the availability of a sufficiently large history of recorded SSH observations used in training. This makes HIDRA2 less reliable for locations with less abundant SSH training data. Furthermore, since the inference requires immediate past SSH measurements at input, forecasts cannot be made during temporary tide gauge failures. We address the aforementioned issues with a new architecture, HIDRA3, that considers observations from multiple locations, shares the geophysical encoder across the locations, and constructs a joint latent state, which is decoded into forecasts at individual locations. The new architecture brings several benefits: (i) it improves training at locations with scarce historical SSH data, (ii) it enables predictions even at locations with sensor failures, and (iii) it reliably estimates prediction uncertainties. HIDRA3 is evaluated by jointly training on eleven tide gauge locations along the Adriatic. Results show that HIDRA3 outperforms HIDRA2 and the standard numerical model NEMO by ~15 % and ~13 % MAE reduction at high SSH values, respectively, setting a solid new state-of-the-art. Furthermore, HIDRA3 shows remarkable performance at substantially smaller amounts of training data compared with HIDRA2, making it appropriate for sea level forecasting in basins with large regional variability in the available tide gauge data.

Received: 08 Jul 2024 – Discussion started: 29 Aug 2024

Publisher's note: Copernicus Publications remains neutral with regard to jurisdictional claims made in the text, published maps, institutional affiliations, or any other geographical representation in this preprint. The responsibility to include appropriate place names lies with the authors.

Download & links

Marko Rus, Hrvoje Mihanović, Matjaž Ličer, and Matej Kristan

Status: closed

RC1:
'Comment on egusphere-2024-2068', Anonymous Referee #1, 16 Sep 2024

Review of HIDRA3: a robust deep-learning model for multi-point ensemble

sea level forecasting.
The paper presents a new version of the HIDRA sea level forecasting model. HIDRA3 is a machine learning model with a deep convolutional architecture. The most important update from version 2 is that the current version uses data not just from the local tide gauge it predicts, but also from neighbouring tide gauges, which allows prediction also when the local tide gauge is not operational.
The paper is well written, figures are nice and the model architecture and modelling choices are well described. I recommend publications after some minor revisions.
General points:
1) The manuscript makes the point that their machine learning model outperforms the numerical ocean model NEMO on SSH prediction. But that is not really what is tested. They do get better results on most metrics than what is seen in the specific NEMO run they compare with. However, the performance in that specific NEMO run, says almost nothing about the capabilities of numerical ocean models in general or even of the NEMO modelling systems capabilities. The SSH performance in the NEMO run they compare with depends on modelling choices (resolution, parametrisations, coordinate systems used. etc.) of which NEMO has very many and of course also on the forcing used to run the NEMO model.
Especially on the forcing side the HIDRA model has a great advantage in this comparison as it is allowed to use tide gauge data, whereas as I understand it the NEMO run they compare with does not assimilate sea level data. I would expect, although I don't know, that HIDRA3 without tide gauge data as inputs would perform worse that the specific NEMO run. Anyway, it should be made more clear in the text that although they outperform this specific Copernicus product it does not really imply much about the capabilities of numerical ocean models in general.
2) The uncertainty quantifications and it's capabilities should be elaborated on more in the manuscript.
3) The models architecture is well described, but the reasons for the modelling choices made is not. Perhaps, much of this information is available in earlier HIDRA papers, but I would like to see more motivations for the different modelling choices.
Specific comments:
L13 I think standard numerical model NEMO is the wrong label.

L44 Goes back to general point 1), these comparisons are very much of apples and oranges

L193 The NEMO setup has to be better described. What NEMO version is used? What forcing is used (also temporal and spatial resolution). What is the vertical and horizontal resolution of the model. What vertical coordinate system is used? Does it have a wave model? Does it have data assimilation? Does it have a minimum depth? Information of that kind is needed to give more context to the different comparisons.

Citation: https://doi.org/10.5194/egusphere-2024-2068-RC1
- AC1: 'Reply on RC1', Marko Rus, 17 Oct 2024
  
  The comment was uploaded in the form of a supplement: https://egusphere.copernicus.org/preprints/2024/egusphere-2024-2068/egusphere-2024-2068-AC1-supplement.pdf
  
  Citation: https://doi.org/10.5194/egusphere-2024-2068-AC1
RC2:
'Comment on egusphere-2024-2068', Anonymous Referee #2, 24 Sep 2024
This study introduces HIDRA3, a deep-learning model developed to estimate multi-point sea levels. It builds upon previous work (Rus et al., 2023), aiming to enhance accuracy and handle missing data. While the research aligns with the scope of this journal and is generally well-written, several unclear aspects need to be addressed before the manuscript can be considered for publication.
Major Comments:
The major concern is the unclear methodology and application conditions for HIDRA3.
Page 5, Line 84: The phrase “detailed manual quality checks” is vague. Does this mean the authors removed data if outliers were detected? How were outliers defined? Is this process feasible in real time? If not, HIDRA3 has only been tested under ideal conditions where manual quality checks have already been applied, which may not be applicable in real-world scenarios. It is essential to clarify what data processing was conducted and if this is possible in real-time. If not, HIDRA3’s performance should be tested on original data without quality checks.

Page 5, Line 85: The manuscript suggests that tide data was predicted in one-year intervals. This may lead to “cheating” by using future data for tide predictions. For instance, predicting water levels on June 1, 2019, might involve tide data that includes water levels from that date. In real-world applications, future water level data would not be available. The authors should revise their approach to ensure that tide predictions are made only using available data, rather than yearly-based data.

Page 5, Line 93: The reason for using ERA5 for training and ECMWF for testing is unclear. If the model trained with ERA5 outperforms the model trained with ECMWF, the authors should clearly present this result. Otherwise, the choice to use ERA5 for training is unjustified.

Page 7, Line 119: HIDRA3 incorporates additional features (sea surface temperature and waves) compared to HIDRA2, but this is not clearly stated. The manuscript should explain why these features were included. Although Section 3.4.2 discusses their impact, it does not analyze their individual contributions. The authors should reference feature selection studies and test the impact of each new feature (sea surface temperature and waves) to justify their inclusion.

Sections 2.2.3 and 2.2.4: It is unclear how missing data (denoted as xi) is handled. Page 9, Line 150 mentions that missing values are estimated from “s,” but it is unclear what “s” refers to. The authors need to clarify what the feature fusion module is doing and explain the difference between xi and s, preferably with a figure for better understanding.

Page 11, Line 191: The differences between HIDRA2, NEMO, and HIDRA3 are not adequately explained. It would be helpful for the authors to provide a clear comparison of these models. For example, HIDRA2 does not consider temperature and wave data. Both HIDRA2 and HIDRA3 are designed for 72-hour predictions. NEMO, on the other hand, performs bias correction every 12 hours. Clarifying these differences would strengthen the manuscript.

Tables 2, 3, and 4: It is unclear why the recall, precision, and F1 scores for “low SSH values” are missing. Low water level predictions are important, particularly for critical infrastructure like nuclear power plants or harbors. The authors should explain why these metrics are missing and include them if possible.

Figure 11: The manuscript only considers “pair” failures for tide gauges, but more realistic scenarios should be explored. For instance, a failure involving multiple tide stations, such as in the northern Adriatic (KP, VE, RA), where water levels show similar trends (as per Figure 2), would offer a more realistic test of HIDRA3’s performance. Testing such scenarios would strengthen the justification for using HIDRA3 over other approaches.

There is no dedicated section on the limitations of HIDRA3. For example, HIDRA3 does not work if data from at least one station is missing for the 72-hour prediction window. The limitations should be clearly stated.

Minor Comments:
Page 3, Line 45: The full name of HIDRA should be provided.

Page 3, Line 60: The authors should expand their literature review to include studies that address missing data in real time, such as Lee and Park (2016) and Vieira et al. (2020), for better context.

Page 5, Line 86-89: It would be helpful to clarify where the “high” and “low” data will be used in the next section. As written, the reason for defining “high” and “low” is unclear.

Figure 2 and Page 5, Line 89: The location names in Figure 2 and the acronyms used in the text should be consistent for readability.

Figure 5 and Page 7, Line 118: The output dimensions are different for various features (e.g., wind and pressure have different dimensions compared to others). The caption for Figure 5 should be corrected to reflect these differences.

Figure 6: The authors should explain why the input dimension is 1152*36, given that the output dimension of Figure 5 is 512*36*1*1. The change in dimensions needs clarification.

Figures 6 and 7: The terms “2X” and “4X” should be explained, as their meaning is unclear.

Page 8, Line 130: The authors mention a “dense layer,” but later refer to “dropout” in Figure 7. This needs clarification, as dropout is not typically associated with fully connected layers.

Section 2.2.3 and 2.2.4: Including a diagram, similar to Figure 6, would help readers understand the concepts better.

Page 9, Line 158: The term “mean” in “SSH mean value prediction” is unclear. The authors should clarify whether this is a typo or explain its meaning.

Page 10, Line 163: The standardization process needs further explanation. Was the data normalized for each case or across the entire dataset?

Page 10, Line 170: For consistency, the term “first stage” should be replaced with “first phase.”

Page 11, Line 193: The full name of NEMO should be provided.

Lee, J. W., & Park, S. C. (2016). Artificial neural network-based data recovery system for the time series of tide stations. Journal of Coastal Research, 32(1), 213-224.

Vieira, F., Cavalcante, G., Campos, E., & Taveira-Pinto, F. (2020). A methodology for data gap filling in wave records using Artificial Neural Networks. Applied Ocean Research, 98, 102109.
Citation: https://doi.org/10.5194/egusphere-2024-2068-RC2
- AC2: 'Reply on RC2', Marko Rus, 17 Oct 2024
  
  The comment was uploaded in the form of a supplement: https://egusphere.copernicus.org/preprints/2024/egusphere-2024-2068/egusphere-2024-2068-AC2-supplement.pdf
  
  Citation: https://doi.org/10.5194/egusphere-2024-2068-AC2

Status: closed

RC1:
'Comment on egusphere-2024-2068', Anonymous Referee #1, 16 Sep 2024

Review of HIDRA3: a robust deep-learning model for multi-point ensemble

sea level forecasting.
The paper presents a new version of the HIDRA sea level forecasting model. HIDRA3 is a machine learning model with a deep convolutional architecture. The most important update from version 2 is that the current version uses data not just from the local tide gauge it predicts, but also from neighbouring tide gauges, which allows prediction also when the local tide gauge is not operational.
The paper is well written, figures are nice and the model architecture and modelling choices are well described. I recommend publications after some minor revisions.
General points:
1) The manuscript makes the point that their machine learning model outperforms the numerical ocean model NEMO on SSH prediction. But that is not really what is tested. They do get better results on most metrics than what is seen in the specific NEMO run they compare with. However, the performance in that specific NEMO run, says almost nothing about the capabilities of numerical ocean models in general or even of the NEMO modelling systems capabilities. The SSH performance in the NEMO run they compare with depends on modelling choices (resolution, parametrisations, coordinate systems used. etc.) of which NEMO has very many and of course also on the forcing used to run the NEMO model.
Especially on the forcing side the HIDRA model has a great advantage in this comparison as it is allowed to use tide gauge data, whereas as I understand it the NEMO run they compare with does not assimilate sea level data. I would expect, although I don't know, that HIDRA3 without tide gauge data as inputs would perform worse that the specific NEMO run. Anyway, it should be made more clear in the text that although they outperform this specific Copernicus product it does not really imply much about the capabilities of numerical ocean models in general.
2) The uncertainty quantifications and it's capabilities should be elaborated on more in the manuscript.
3) The models architecture is well described, but the reasons for the modelling choices made is not. Perhaps, much of this information is available in earlier HIDRA papers, but I would like to see more motivations for the different modelling choices.
Specific comments:
L13 I think standard numerical model NEMO is the wrong label.

L44 Goes back to general point 1), these comparisons are very much of apples and oranges

L193 The NEMO setup has to be better described. What NEMO version is used? What forcing is used (also temporal and spatial resolution). What is the vertical and horizontal resolution of the model. What vertical coordinate system is used? Does it have a wave model? Does it have data assimilation? Does it have a minimum depth? Information of that kind is needed to give more context to the different comparisons.

Citation: https://doi.org/10.5194/egusphere-2024-2068-RC1
- AC1: 'Reply on RC1', Marko Rus, 17 Oct 2024
  
  The comment was uploaded in the form of a supplement: https://egusphere.copernicus.org/preprints/2024/egusphere-2024-2068/egusphere-2024-2068-AC1-supplement.pdf
  
  Citation: https://doi.org/10.5194/egusphere-2024-2068-AC1
RC2:
'Comment on egusphere-2024-2068', Anonymous Referee #2, 24 Sep 2024
This study introduces HIDRA3, a deep-learning model developed to estimate multi-point sea levels. It builds upon previous work (Rus et al., 2023), aiming to enhance accuracy and handle missing data. While the research aligns with the scope of this journal and is generally well-written, several unclear aspects need to be addressed before the manuscript can be considered for publication.
Major Comments:
The major concern is the unclear methodology and application conditions for HIDRA3.
Page 5, Line 84: The phrase “detailed manual quality checks” is vague. Does this mean the authors removed data if outliers were detected? How were outliers defined? Is this process feasible in real time? If not, HIDRA3 has only been tested under ideal conditions where manual quality checks have already been applied, which may not be applicable in real-world scenarios. It is essential to clarify what data processing was conducted and if this is possible in real-time. If not, HIDRA3’s performance should be tested on original data without quality checks.

Page 5, Line 85: The manuscript suggests that tide data was predicted in one-year intervals. This may lead to “cheating” by using future data for tide predictions. For instance, predicting water levels on June 1, 2019, might involve tide data that includes water levels from that date. In real-world applications, future water level data would not be available. The authors should revise their approach to ensure that tide predictions are made only using available data, rather than yearly-based data.

Page 5, Line 93: The reason for using ERA5 for training and ECMWF for testing is unclear. If the model trained with ERA5 outperforms the model trained with ECMWF, the authors should clearly present this result. Otherwise, the choice to use ERA5 for training is unjustified.

Page 7, Line 119: HIDRA3 incorporates additional features (sea surface temperature and waves) compared to HIDRA2, but this is not clearly stated. The manuscript should explain why these features were included. Although Section 3.4.2 discusses their impact, it does not analyze their individual contributions. The authors should reference feature selection studies and test the impact of each new feature (sea surface temperature and waves) to justify their inclusion.

Sections 2.2.3 and 2.2.4: It is unclear how missing data (denoted as xi) is handled. Page 9, Line 150 mentions that missing values are estimated from “s,” but it is unclear what “s” refers to. The authors need to clarify what the feature fusion module is doing and explain the difference between xi and s, preferably with a figure for better understanding.

Page 11, Line 191: The differences between HIDRA2, NEMO, and HIDRA3 are not adequately explained. It would be helpful for the authors to provide a clear comparison of these models. For example, HIDRA2 does not consider temperature and wave data. Both HIDRA2 and HIDRA3 are designed for 72-hour predictions. NEMO, on the other hand, performs bias correction every 12 hours. Clarifying these differences would strengthen the manuscript.

Tables 2, 3, and 4: It is unclear why the recall, precision, and F1 scores for “low SSH values” are missing. Low water level predictions are important, particularly for critical infrastructure like nuclear power plants or harbors. The authors should explain why these metrics are missing and include them if possible.

Figure 11: The manuscript only considers “pair” failures for tide gauges, but more realistic scenarios should be explored. For instance, a failure involving multiple tide stations, such as in the northern Adriatic (KP, VE, RA), where water levels show similar trends (as per Figure 2), would offer a more realistic test of HIDRA3’s performance. Testing such scenarios would strengthen the justification for using HIDRA3 over other approaches.

There is no dedicated section on the limitations of HIDRA3. For example, HIDRA3 does not work if data from at least one station is missing for the 72-hour prediction window. The limitations should be clearly stated.

Minor Comments:
Page 3, Line 45: The full name of HIDRA should be provided.

Page 3, Line 60: The authors should expand their literature review to include studies that address missing data in real time, such as Lee and Park (2016) and Vieira et al. (2020), for better context.

Page 5, Line 86-89: It would be helpful to clarify where the “high” and “low” data will be used in the next section. As written, the reason for defining “high” and “low” is unclear.

Figure 2 and Page 5, Line 89: The location names in Figure 2 and the acronyms used in the text should be consistent for readability.

Figure 5 and Page 7, Line 118: The output dimensions are different for various features (e.g., wind and pressure have different dimensions compared to others). The caption for Figure 5 should be corrected to reflect these differences.

Figure 6: The authors should explain why the input dimension is 1152*36, given that the output dimension of Figure 5 is 512*36*1*1. The change in dimensions needs clarification.

Figures 6 and 7: The terms “2X” and “4X” should be explained, as their meaning is unclear.

Page 8, Line 130: The authors mention a “dense layer,” but later refer to “dropout” in Figure 7. This needs clarification, as dropout is not typically associated with fully connected layers.

Section 2.2.3 and 2.2.4: Including a diagram, similar to Figure 6, would help readers understand the concepts better.

Page 9, Line 158: The term “mean” in “SSH mean value prediction” is unclear. The authors should clarify whether this is a typo or explain its meaning.

Page 10, Line 163: The standardization process needs further explanation. Was the data normalized for each case or across the entire dataset?

Page 10, Line 170: For consistency, the term “first stage” should be replaced with “first phase.”

Page 11, Line 193: The full name of NEMO should be provided.

Lee, J. W., & Park, S. C. (2016). Artificial neural network-based data recovery system for the time series of tide stations. Journal of Coastal Research, 32(1), 213-224.

Vieira, F., Cavalcante, G., Campos, E., & Taveira-Pinto, F. (2020). A methodology for data gap filling in wave records using Artificial Neural Networks. Applied Ocean Research, 98, 102109.
Citation: https://doi.org/10.5194/egusphere-2024-2068-RC2
- AC2: 'Reply on RC2', Marko Rus, 17 Oct 2024
  
  The comment was uploaded in the form of a supplement: https://egusphere.copernicus.org/preprints/2024/egusphere-2024-2068/egusphere-2024-2068-AC2-supplement.pdf
  
  Citation: https://doi.org/10.5194/egusphere-2024-2068-AC2

Marko Rus, Hrvoje Mihanović, Matjaž Ličer, and Matej Kristan

Data sets

Training and Test Datasets, Pretrained Weights and Predictions for HIDRA3 Marko Rus, Hrvoje Mihanović, Matjaž Ličer, and Matej Kristan https://doi.org/10.5281/zenodo.12571170

Model code and software

Code for HIDRA3: A Robust Deep-Learning Model for Multi-Point Sea-Surface Height Forecasting Marko Rus, Hrvoje Mihanović, Matjaž Ličer, and Matej Kristan https://doi.org/10.5281/zenodo.12570449

Marko Rus, Hrvoje Mihanović, Matjaž Ličer, and Matej Kristan

Viewed

Total article views: 382 (including HTML, PDF, and XML)

HTML	PDF	XML	Total	BibTeX	EndNote
200	51	131	382	7	8

HTML: 200
PDF: 51
XML: 131
Total: 382
BibTeX: 7
EndNote: 8

Views and downloads (calculated since 29 Aug 2024)

Month	HTML	PDF	XML	Total
Aug 2024	38	12	3	53
Sep 2024	57	18	45	120
Oct 2024	46	10	29	85
Nov 2024	36	4	49	89
Dec 2024	23	7	5	35

Cumulative views and downloads (calculated since 29 Aug 2024)

Month	HTML	PDF	XML	Total
Aug 2024	38	12	3	53
Sep 2024	57	18	45	120
Oct 2024	46	10	29	85
Nov 2024	36	4	49	89
Dec 2024	23	7	5	35

Viewed (geographical distribution)

Total article views: 384 (including HTML, PDF, and XML) Thereof 384 with geography defined and 0 with unknown origin.

Country	#	Views	%

Latest update: 30 Dec 2024

Short summary

HIDRA3 is a novel deep-learning model for predicting sea levels and storm surges, offering significant improvements over previous models and numerical simulations. It utilizes data from multiple tide gauges, enhancing predictions even with limited historical data and during sensor outages. With its advanced architecture, HIDRA3 outperforms the current state-of-the-art models by achieving up to 15 % lower mean absolute error, proving effective for coastal flood forecasting in diverse conditions.


Total:	0
HTML:	0
PDF:	0
XML:	0