the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
| 20 Feb 2024
Deep Learning for Super-Resolution of Mediterranean Sea Surface Temperature Fields
Abstract. Sea surface temperature (SST) is one of the essential variables of the Earth climate system. Being at the interface with the atmosphere, SST modulates heat fluxes in and out of the ocean, provides insight on several upper/interior ocean dynamical processes, and it is a fundamental indicator of climate variability potentially impacting marine ecosystems’ health. Its accurate estimation and regular monitoring from space is therefore crucial. However, even if satellite infrared/microwave measurements provide a much better coverage than what achievable from in situ platforms, they cannot sense the sea surface under cloudy/rainy conditions. Large gaps are present even in merged multi-sensor satellite products and different statistical strategies have thus been proposed to obtain gap-free (L4) images, mostly based on the Optimal Interpolation algorithms. This kind of techniques, however, filter out the signals below the space-time decorrelation scales considered, significantly smoothing most of the small mesoscale and submesoscale features. Here, deep learning models, originally designed for single image Super Resolution (SR), are applied to enhance the effective resolution of SST products and the accuracy of SST gradients. SR schemes include a set of computer vision techniques leveraging Convolutional Neural Networks to retrieve high-resolution data from low-resolution images. A dilated convolutional multi-scale learning network, which includes an adaptive residual strategy and implements a channel attention mechanism, is used to reconstruct features in SST data at 1/100° spatial resolution starting from 1/16° data over the Mediterranean Sea. The application of this technique shows a remarkable improvement in the high resolution reconstruction, being able to capture small scale features and providing a root-mean-squared-difference improvement of 0.02 °C with respect to the L3 ground-truth data.
-
Notice on discussion status
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
-
Preprint
(14284 KB)
-
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
- Preprint
(14284 KB) - Metadata XML
- BibTeX
- EndNote
- Final revised paper
Journal article(s) based on this preprint
Interactive discussion
Status: closed
-
RC1: 'Comment on egusphere-2024-455', Anonymous Referee #1, 22 Feb 2024
The article presents a study on improving the resolution of Sea Surface Temperature (SST) fields in the Mediterranean Sea using deep learning models, specifically a dilated convolutional multi-scale learning network. This approach allows for better capture of small scale features and gradients in SST data, overcoming limitations of traditional satellite-based measurements and interpolation methods. The study demonstrates significant improvements in the accuracy and resolution of SST reconstructions, highlighting the potential of deep learning in enhancing oceanographic data analysis and climate research. But the experiment needs some work.
- Incorporate additional independent datasets for validating the improved SST fields, ensuring the model's robustness across various conditions and regions within the Mediterranean Sea.
- Compare the performance of the proposed deep learning model against existing other deep learning super-resolution models, such as GAN series, providing a comprehensive analysis of its advantages and limitations.
- Conduct a sensitivity analysis to understand the impact of different parameters within the dilated convolutional multi-scale learning network, optimizing the model's performance.
- Could the article be enriched by including a paragraph discussing how high-resolution SST fields can be incorporated into regional climate models to improve the accuracy of climate projections in the Mediterranean region?
- AC1: 'Reply on RC1', Claudia Fanelli, 14 May 2024
-
RC2: 'Comment on egusphere-2024-455', Anonymous Referee #2, 29 Mar 2024
This study aims at reconstructing small scales SST from a low resolution SST field and provides a gap free L4 dataset resolving scales up to 5 km using deep learning algorithms. The super Resolution Convolution Network is learning using an ensemble of low resolution and high resolution SST images in the Mediterranean Sea. Results are shown for one snapshot and analyses are shown for cloud-free areas. Compared to the first guess at low resolution, the SST field has indeed been improved but this study needs further investigation and discussion of the results.
1 - Comparison with other High resolution L4 products are required as the first guess is at very low resolution and does not reflect what is currently available (such as MUR product from JPL for example). Some SST products perform well when the cloud coverage is small but have a resolved spatial scale that varies a lot with the cloud cover. Comparison of the different SST products on a cloud-free event and on a cloudy day will show the potential of this new SST product much more homogeneous in time.
2- From the snapshot with the SST gradient, One wonders if the method enables the reconstruction of smaller scales or is more of a gradient enhancement. It would be helpful to comment on that and illustrate if submesoscales are really generated in this SST reconstruction.
3 - The added value of this SST product is not clear from Table 1 and the spectrum figure (Fig.6). I would recommend adding other SST high resolution L4 products as discussed in point 4. Regarding the SSIM index, it may be interesting to also discuss the details of the decomposition between contrast / structures and luminance if the results prove to be relevant (the contrast and structure are expected to be much more improved than the luminance).
4- Finally, to really describe the effective resolution, the study of the ratio between the spectral content of the reconstructed data and the truth is more relevant than mere spectrum (as detailed in Ballarotta, et al 2019.). It should be included in a further analyses of this SST product to comment on the effective resolution of the SST product
NB: Ballarota et al 2019: Ballarotta, M., Ubelmann, C., Pujol, M.-I., Taburet, G., Fournier, F., Legeais, J.-F., Faugère, Y., Delepoulle, A., Chelton, D., Dibarboure, G., and Picot, N.: On the resolutions of ocean altimetry maps, Ocean Sci., 15, 1091–1109, https://doi.org/10.5194/os-15-1091-2019, 2019.
Citation: https://doi.org/10.5194/egusphere-2024-455-RC2 - AC2: 'Reply on RC2', Claudia Fanelli, 14 May 2024
-
RC3: 'Comment on egusphere-2024-455', Anonymous Referee #3, 02 Apr 2024
In their work the authors are tackling an important problem in geophysics, namely the reconstruction of high resolution fields of Sea Surface Temperature from partial high-resolution observations. Super-resolution is an ill posed problem, given that identical low-resolution fields can correspond to different high-resolution fields. The authors adapt their previous work on ADT to SST field reconstructions, as well as their well established knowledge of SST field reconstruction, showing improvements over the Mediterranean basin, within the confines of their experiment.
I really appreciated many parts of the article, notably the varying metrics and case studies to evaluate the quality of the reconstructions.
As it stands, I have some major and minor criticisms for the article that, should the authors address, would make for a significant contribution to the community.
The validation process is prone to data leakage. 4 days out of a year of data were omitted, but there is no mention of removing some days before or some days after in order to prevent data leakage. The physical reasoning of this is absent. Are the structures that decorelated after one day given the removal of the 200km smoothed field?
In general the L119 statement: “The test dataset is finally selected separating the 15% of the tiles available after 120 the preprocessing, chosen in order to be able to reconstruct the full geographical coverage of four days which are representative of different seasons.” requires clarification. I read it as 4 individual days, one in each season. It could be understood as patches covering the whole area, spread out over each season. I would expect to have more of a cross-validation approach given the one year dataset limitation.Another major concern is the input. The input, presently, is only the first guess (removing a sliding window), and the information coming from the L3 satellite product is not used as a complimentary input. Why did the authors deprived themselves from potential additional input such as multiple time steps and L3 products? Other works (such as Archambault et al, Martin et al) in SSH fields have training procedures where some of the satellite information is omitted from the target in order to validate the approach.
There is no mention of how the total field reconstruction over the whole Mediterranean sea is output. If the image was made by recomposing a sliding window reconstruction that should be mentioned. Given that the network learns filters, one could conceivably apply them on the whole image, though I expect the attention layers to pose an issue.
The choice of architecture, while documented, is not justified. Were the hyper-parameters optimized? Other architectures evaluated? The authors mention a lot of competing methods, but do not compare their architecture to them. (DINEOF, DINCAE, to cite but two) Are the computational and expertise cost justified versus other methods? The results seem to indicate a 0.02°C improvement; is the architecture stable through different initializations? The method section (2.2) seems to assume unfamiliarity with neural networks, providing intuitive explanations for basic architectural blocks, but then very quickly skims over important details of the more complicated blocks of the architecture. This part would benefit reducing the initial explanation of activation functions and CNNs (such as the interpretation of lines 59 to 61 which is intuitive but could easily not correspond to the exact explanations provided given the non-linear activations) and expanding on the reasoning of the architectural choices (the adaptive part of the ARB is not discussed, implying the rest of the architecture is non adaptive).
No mention is made to VIT and diffusion-based super resolution techniques that have become state of the art in computer vision. I can understand the daunting nature of these, but should they not be mentioned as potential further steps, at least? The latter is especially significant: the field reconstructions obtained through optimizing RMSE favor smoothness, and often do not represent physically feasible oceanic states. Graphcast for example has been abandoned in favor of Gencast for that very reason. Given that the model is in NRT, and therefore would be used for constraining operational models, it might be interesting to at least think about this. It is even more important given the non-bijective nature of the problem.Fig.1 would benefit from locating with a bounding box or three the patches on the right hand side.
182: max(I) in denormalized space i.e. K°? Or in the space where the large field is removed 200km? Is it computed over the patches, or the whole Mediterranean?
Citation: https://doi.org/10.5194/egusphere-2024-455-RC3 - AC3: 'Reply on RC3', Claudia Fanelli, 14 May 2024
-
RC4: 'Comment on egusphere-2024-455', Peter Cornillon, 09 Apr 2024
This manuscript explores the use of a deep learning model to enhance the spatial resolution of L4 SST products in regions where missing data, generally associated with cloud cover, results in coarse fields obtained with objective analysis techniques. As I understand it, the model, which the authors have developed, dADR-SR, is trained with high resolution (HR, 1/16 degree) input fields and ultra high resolution (UHR 1/100 degree) target fields. It is then applied to HR fields and shown to reproduce structure at very nearly the same spatial resolution as test (1/100 degree) fields.
The ML model they have developed appears to perform very well for the test dataset they use but I struggled with the manuscript and I believe that it needs a fair amount of editorial work before it is ready for publication. Specifically, after reading the manuscript several times, I think that I sorted out what was done but, I must admit that I am still not sure that I have it right. I’ve included a figure in which I have tried to show the datasets, which I think they are using, and how these dataset relate to one another. First, there is a set of four datasets, two high resolution (1/16 degree), an L3S and an L4, and two ultra high resolution (1/100 degree) again an L3S and an L4. A subset of these datasets are used to train the dADR-SR algorithm and the HR L4 dataset is then fed into the trained model and the output is compared with an L3S UHR dataset (i.e., one built using the same algorithm as used to build the standard L3S UHR products used to train) constructed with SLSTR data from the Sentinel 3A and 3B satellites. I don’t think that the SLSTR data are used in the construction of the standard products but I may have that wrong, well, I may have all of this wrong, for which I apologize. Adding to the confusion is that the authors appear to have changed the terminology they use for the datasets. In the abstract and in the Discussion section the authors refer to low resolution (LR) and high resolution (HR) datasets while in the remainder of the document they refer to high resolution (HR) and ultra high resolution (UHR) datasets. I’m guessing that LR (used in the Introduction and Discussion) is what they later refer to as HR and HR (in the Intro and Discussion) is what they later refer to as UHR.
Bottom line: I believe that a bit more description of what goes into the standard datasets that are used to train and later as input to the dADR-SR model, along with a clear description of differences, if any, between the datasets produced as input and/or evaluation for the work undertaken in this study. I also think that a figure showing the relationship of the datasets and processing steps, at a very gross level—sort of like the figure that I have attempted to put together below—would go a long way to making the manuscript easier to follow.
In addition to the general concern outlined above I have made a number of editorial suggestions, which I hope will help to make the manuscript a bit easier to read. These are included in the attached manuscript either as hand-written annotations or as typed comments.
Finally, I would like to apologize to the authors for the length of time that I took for this review—I had another manuscript, which I was asked to review, and, which had to be completed before addressing this one as well as some family issues.
Peter Cornillon
- AC4: 'Reply on RC4', Claudia Fanelli, 14 May 2024
Interactive discussion
Status: closed
-
RC1: 'Comment on egusphere-2024-455', Anonymous Referee #1, 22 Feb 2024
The article presents a study on improving the resolution of Sea Surface Temperature (SST) fields in the Mediterranean Sea using deep learning models, specifically a dilated convolutional multi-scale learning network. This approach allows for better capture of small scale features and gradients in SST data, overcoming limitations of traditional satellite-based measurements and interpolation methods. The study demonstrates significant improvements in the accuracy and resolution of SST reconstructions, highlighting the potential of deep learning in enhancing oceanographic data analysis and climate research. But the experiment needs some work.
- Incorporate additional independent datasets for validating the improved SST fields, ensuring the model's robustness across various conditions and regions within the Mediterranean Sea.
- Compare the performance of the proposed deep learning model against existing other deep learning super-resolution models, such as GAN series, providing a comprehensive analysis of its advantages and limitations.
- Conduct a sensitivity analysis to understand the impact of different parameters within the dilated convolutional multi-scale learning network, optimizing the model's performance.
- Could the article be enriched by including a paragraph discussing how high-resolution SST fields can be incorporated into regional climate models to improve the accuracy of climate projections in the Mediterranean region?
- AC1: 'Reply on RC1', Claudia Fanelli, 14 May 2024
-
RC2: 'Comment on egusphere-2024-455', Anonymous Referee #2, 29 Mar 2024
This study aims at reconstructing small scales SST from a low resolution SST field and provides a gap free L4 dataset resolving scales up to 5 km using deep learning algorithms. The super Resolution Convolution Network is learning using an ensemble of low resolution and high resolution SST images in the Mediterranean Sea. Results are shown for one snapshot and analyses are shown for cloud-free areas. Compared to the first guess at low resolution, the SST field has indeed been improved but this study needs further investigation and discussion of the results.
1 - Comparison with other High resolution L4 products are required as the first guess is at very low resolution and does not reflect what is currently available (such as MUR product from JPL for example). Some SST products perform well when the cloud coverage is small but have a resolved spatial scale that varies a lot with the cloud cover. Comparison of the different SST products on a cloud-free event and on a cloudy day will show the potential of this new SST product much more homogeneous in time.
2- From the snapshot with the SST gradient, One wonders if the method enables the reconstruction of smaller scales or is more of a gradient enhancement. It would be helpful to comment on that and illustrate if submesoscales are really generated in this SST reconstruction.
3 - The added value of this SST product is not clear from Table 1 and the spectrum figure (Fig.6). I would recommend adding other SST high resolution L4 products as discussed in point 4. Regarding the SSIM index, it may be interesting to also discuss the details of the decomposition between contrast / structures and luminance if the results prove to be relevant (the contrast and structure are expected to be much more improved than the luminance).
4- Finally, to really describe the effective resolution, the study of the ratio between the spectral content of the reconstructed data and the truth is more relevant than mere spectrum (as detailed in Ballarotta, et al 2019.). It should be included in a further analyses of this SST product to comment on the effective resolution of the SST product
NB: Ballarota et al 2019: Ballarotta, M., Ubelmann, C., Pujol, M.-I., Taburet, G., Fournier, F., Legeais, J.-F., Faugère, Y., Delepoulle, A., Chelton, D., Dibarboure, G., and Picot, N.: On the resolutions of ocean altimetry maps, Ocean Sci., 15, 1091–1109, https://doi.org/10.5194/os-15-1091-2019, 2019.
Citation: https://doi.org/10.5194/egusphere-2024-455-RC2 - AC2: 'Reply on RC2', Claudia Fanelli, 14 May 2024
-
RC3: 'Comment on egusphere-2024-455', Anonymous Referee #3, 02 Apr 2024
In their work the authors are tackling an important problem in geophysics, namely the reconstruction of high resolution fields of Sea Surface Temperature from partial high-resolution observations. Super-resolution is an ill posed problem, given that identical low-resolution fields can correspond to different high-resolution fields. The authors adapt their previous work on ADT to SST field reconstructions, as well as their well established knowledge of SST field reconstruction, showing improvements over the Mediterranean basin, within the confines of their experiment.
I really appreciated many parts of the article, notably the varying metrics and case studies to evaluate the quality of the reconstructions.
As it stands, I have some major and minor criticisms for the article that, should the authors address, would make for a significant contribution to the community.
The validation process is prone to data leakage. 4 days out of a year of data were omitted, but there is no mention of removing some days before or some days after in order to prevent data leakage. The physical reasoning of this is absent. Are the structures that decorelated after one day given the removal of the 200km smoothed field?
In general the L119 statement: “The test dataset is finally selected separating the 15% of the tiles available after 120 the preprocessing, chosen in order to be able to reconstruct the full geographical coverage of four days which are representative of different seasons.” requires clarification. I read it as 4 individual days, one in each season. It could be understood as patches covering the whole area, spread out over each season. I would expect to have more of a cross-validation approach given the one year dataset limitation.Another major concern is the input. The input, presently, is only the first guess (removing a sliding window), and the information coming from the L3 satellite product is not used as a complimentary input. Why did the authors deprived themselves from potential additional input such as multiple time steps and L3 products? Other works (such as Archambault et al, Martin et al) in SSH fields have training procedures where some of the satellite information is omitted from the target in order to validate the approach.
There is no mention of how the total field reconstruction over the whole Mediterranean sea is output. If the image was made by recomposing a sliding window reconstruction that should be mentioned. Given that the network learns filters, one could conceivably apply them on the whole image, though I expect the attention layers to pose an issue.
The choice of architecture, while documented, is not justified. Were the hyper-parameters optimized? Other architectures evaluated? The authors mention a lot of competing methods, but do not compare their architecture to them. (DINEOF, DINCAE, to cite but two) Are the computational and expertise cost justified versus other methods? The results seem to indicate a 0.02°C improvement; is the architecture stable through different initializations? The method section (2.2) seems to assume unfamiliarity with neural networks, providing intuitive explanations for basic architectural blocks, but then very quickly skims over important details of the more complicated blocks of the architecture. This part would benefit reducing the initial explanation of activation functions and CNNs (such as the interpretation of lines 59 to 61 which is intuitive but could easily not correspond to the exact explanations provided given the non-linear activations) and expanding on the reasoning of the architectural choices (the adaptive part of the ARB is not discussed, implying the rest of the architecture is non adaptive).
No mention is made to VIT and diffusion-based super resolution techniques that have become state of the art in computer vision. I can understand the daunting nature of these, but should they not be mentioned as potential further steps, at least? The latter is especially significant: the field reconstructions obtained through optimizing RMSE favor smoothness, and often do not represent physically feasible oceanic states. Graphcast for example has been abandoned in favor of Gencast for that very reason. Given that the model is in NRT, and therefore would be used for constraining operational models, it might be interesting to at least think about this. It is even more important given the non-bijective nature of the problem.Fig.1 would benefit from locating with a bounding box or three the patches on the right hand side.
182: max(I) in denormalized space i.e. K°? Or in the space where the large field is removed 200km? Is it computed over the patches, or the whole Mediterranean?
Citation: https://doi.org/10.5194/egusphere-2024-455-RC3 - AC3: 'Reply on RC3', Claudia Fanelli, 14 May 2024
-
RC4: 'Comment on egusphere-2024-455', Peter Cornillon, 09 Apr 2024
This manuscript explores the use of a deep learning model to enhance the spatial resolution of L4 SST products in regions where missing data, generally associated with cloud cover, results in coarse fields obtained with objective analysis techniques. As I understand it, the model, which the authors have developed, dADR-SR, is trained with high resolution (HR, 1/16 degree) input fields and ultra high resolution (UHR 1/100 degree) target fields. It is then applied to HR fields and shown to reproduce structure at very nearly the same spatial resolution as test (1/100 degree) fields.
The ML model they have developed appears to perform very well for the test dataset they use but I struggled with the manuscript and I believe that it needs a fair amount of editorial work before it is ready for publication. Specifically, after reading the manuscript several times, I think that I sorted out what was done but, I must admit that I am still not sure that I have it right. I’ve included a figure in which I have tried to show the datasets, which I think they are using, and how these dataset relate to one another. First, there is a set of four datasets, two high resolution (1/16 degree), an L3S and an L4, and two ultra high resolution (1/100 degree) again an L3S and an L4. A subset of these datasets are used to train the dADR-SR algorithm and the HR L4 dataset is then fed into the trained model and the output is compared with an L3S UHR dataset (i.e., one built using the same algorithm as used to build the standard L3S UHR products used to train) constructed with SLSTR data from the Sentinel 3A and 3B satellites. I don’t think that the SLSTR data are used in the construction of the standard products but I may have that wrong, well, I may have all of this wrong, for which I apologize. Adding to the confusion is that the authors appear to have changed the terminology they use for the datasets. In the abstract and in the Discussion section the authors refer to low resolution (LR) and high resolution (HR) datasets while in the remainder of the document they refer to high resolution (HR) and ultra high resolution (UHR) datasets. I’m guessing that LR (used in the Introduction and Discussion) is what they later refer to as HR and HR (in the Intro and Discussion) is what they later refer to as UHR.
Bottom line: I believe that a bit more description of what goes into the standard datasets that are used to train and later as input to the dADR-SR model, along with a clear description of differences, if any, between the datasets produced as input and/or evaluation for the work undertaken in this study. I also think that a figure showing the relationship of the datasets and processing steps, at a very gross level—sort of like the figure that I have attempted to put together below—would go a long way to making the manuscript easier to follow.
In addition to the general concern outlined above I have made a number of editorial suggestions, which I hope will help to make the manuscript a bit easier to read. These are included in the attached manuscript either as hand-written annotations or as typed comments.
Finally, I would like to apologize to the authors for the length of time that I took for this review—I had another manuscript, which I was asked to review, and, which had to be completed before addressing this one as well as some family issues.
Peter Cornillon
- AC4: 'Reply on RC4', Claudia Fanelli, 14 May 2024
Peer review completion
Journal article(s) based on this preprint
Viewed
| HTML | XML | Total | BibTeX | EndNote | |
|---|---|---|---|---|---|
| 437 | 204 | 41 | 682 | 18 | 17 |
- HTML: 437
- PDF: 204
- XML: 41
- Total: 682
- BibTeX: 18
- EndNote: 17
Viewed (geographical distribution)
| Country | # | Views | % |
|---|
| Total: | 0 |
| HTML: | 0 |
| PDF: | 0 |
| XML: | 0 |
- 1
Claudia Fanelli
Daniele Ciani
Andrea Pisano
Bruno Buongiorno Nardelli
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
- Preprint
(14284 KB) - Metadata XML