the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
| 29 Apr 2024
Combining the U-Net model and a Multi-textRG algorithm for fine SAR ice-water classification
Abstract. Sea ice classification faces challenges due to the similarity among surfaces such as wind-driven open water (OW), smooth thin ice, level first-year ice (FYI), and melted ice surfaces. Previous algorithms combine unsupervised region segmentation and supervised neural networks, yet struggle due to limited manual labels and inaccurate region segmentation. We propose to adopt a supervised neural network followed by a region segmentation algorithm with experiential knowledge involved to solve the ambiguous recognition question and sample number limitation. Provided by the AI4Arctic competition, the preprocessed GCOM-W1 AMSR2 36.5GHz H polarization and Sentinel-1 SAR EW dual-polarization data, the CIS/DMI ice chart labels, and the pre-trained U-Net CNN model are employed to perform semantic segmentation of ice and water with near-100 % accuracy. Subsequently, within the U-Net semantically segmented ice mask, a multistage pixel-based ice detection algorithm developed on GLCM textures of SAR images and region growing approach, the Multi-textRG algorithm, refines the ice edge details. We validate the results on Landsat-8 and Sentinel-2 optical data yielding an overall accuracy of 83.11 %, low false negative (FN) of 4.03 % indicating underestimated low backscatter ice surfaces and higher false positive (FP) of 12.86 % reflecting their resolution difference along edges. More importantly, we fused the SAR-based ice detection with CIS/DMI ice charts and AMSR2 ASI SIC product obtaining SAR-Chart and SAR-AMSR2 labels, which enhance ice edge depictions and SIC variation contours. Repeating the two-step procedure with the high-precision SIC labels demonstrates the U-Net model's capability to extract detailed ice edges information and stability of the Multi-textRG algorithm. The U-Net model trained on SAR-AMSR2 label achieves the highest R2-score of 91.993 %, the largest OWrecall (recall of OW) of 99.268 %, and large ov40recall (recall of ice with over-40 % SIC) of 99.207 %. Our algorithm framework solves the accurate ice-water classification at all seasons and facilitates the sample labelling for improving SIC estimation accuracy based on CNN models.
- Preprint
(3719 KB) - Metadata XML
-
Supplement
(30349 KB) - BibTeX
- EndNote
Status: final response (author comments only)
-
RC1: 'Comment on egusphere-2024-1177', Anton Korosov, 06 Jun 2024
The manuscript presents a complex set of experiments with U-net convolutional neural networks, texture features and fusion techniques aiming at predicting accurate sea ice concentration retrieval from SAR and PMW data. Despite several novel ideas contained in the manuscript I would recommend rejecting it. The reasons are specified in the attached PDF file.
-
AC1: 'Reply on RC1', Yan Sun, 06 Jun 2024
Reply to the review from Anton Korosov for our manuscript “Combining the U-Net model and a Multi-textRG algorithm for fine SAR ice-water classification” by Yan Sun, Shaoyin Wang, Xiao Cheng, Teng Li, Chong Liu, Yufang Ye, Xi Zhao
Thanks for your valuable review and kindly comments. We admit the ambition to comprehensively solve the problem of ice-water classification and thus made a long paper. Though we have tried to build the logical structure by separating the description of U-net model and the Multi-textRG algorithm in methods and results sections, e.g., only Section 3.1 and Section 4.1 is used to describe the U-net model settings and predictions. The reading feedback from Dr. Korosov as “the second reason” is very valuable and accurate. We appreciate that Dr. Korosov proposed some detailed words organization and expression suggestions at the last paragraph, which are great tricks to further improve our manuscript.
In the following, we will reply in two aspects: 1. Is the chain of algorithm is suitable? 2. Do we need to split the manuscript into two logically separable?
1) The manuscript aims to: 1. Proposing a classification algorithm, U-net (one) + Multi-textRG algorithm, to achieve the high resolution/high precision ice and water classification; 2. After achieving the high resolution /high precision ice and water classification, we fused the “SAR growing ice” results (i.e., the classification results) with ice charts and AMSR2 ASI SIC respectively to acquire more detailed training labels. No matter how it is processed, the results are considered as three independent labels: ice charts (Chart SIC), SAR-Chart SIC, and SAR-AMSR SIC. Then evaluating the ability of U-net model for detail prediction using these labels. Dividing SAR-Chart SIC label into training, validation and test datasets, the U-net two was trained on training dataset, the validation metrics (R2 and 3 recall values) were calculated on validation dataset, and the test metrics (R2 and 3 recall values) were calculated on test datasets. The U-net three was also trained, validated, and tested on SAR-AMSR2 SIC label in this way.
The chain of algorithm is:
1. The “U-net one” and the “Multi-textRG” algorithm process S1 SAR data, trained on the AI4Arctic ice charts, and generate “SAR growing ice” product.
2. The “SAR growing ice” is fused with the AI4Arctic ice charts to produce “SAR-Chart SIC”. Similar for SAR-AMSR2 SIC.
3. The prediction from the U-net trained on “SAR-Chart SIC” is validated against “SAR-Chart SIC” (U-net two). Same for SAR-AMSR2 SIC (U-net three).
(“SAR growing ice” , we hope to correct it in the revision.)
Within it, the U-net one can achieve semantic segmentation for coarse ice regions and the Multi-textRG algorithm detect ice edge details in high precision within the U-net segmented ice regions. The Multi-textRG algorithm is designed by selecting and combining GLCM textures based on some approaches and experiences. Without Multi-textRG algorithm, there are not high precision “SAR growing ice” results and new data-fused labels and further experiments of U-net two and three.
For the chain of algorithm, the only input includes Sentinel-1 HH and HV polarized images, AMSR2 36.5GHz H-polarization data, the AI4Arctic ice charts label. The only output is binary ice-water classification.
The U-net one + Multi-textRG algorithm can be used for high-precision ice-water classification, whereas the complex procedure may limit its operational usage. Otherwise, the Multi-textRG algorithm results can be used as training labels, and we demonstrated this: using the above-mentioned inputs and training the U-net model twice (U-net one and U-net two), we achieved high-precision and detailed ice-water classification from U-net prediction (see Fig. 7(e)). Also, using the inputs including Sentinel-1 HH and HV polarized images, AMSR2 36.5GHz H-polarization data, and the AMSR2 ASI SIC label and training the U-net model twice (like U-net one and U-net three), we could achieve better ice-water classification from U-net predict (see Fig. 7(f)). From the overall perspective, we think the chain of algorithm is reasonable.
The paper focus is always the Multi-textRG algorithm. We have always concentrated the overall goal of this paper on solving the questions for fine classification of ice and water in the Arctic. We noticed the extreme condition for R2 calculation, that is, if the predict results is all 0% SIC, the R2 value can be 100%. This has nothing to do with labels. However, we used the same U-net model developed and provided by the AI4Arctic project. We do not focus on the U-net model design or model evaluation, thus we keep it the same as much as possible, limit the descriptions in Section 3.1 and Section 4.1. To avoiding the extreme 100% R2, we therefore introduced OWrecall, ov40recall and bl40recall metrics to depict the predict ability of U-net for binary ice and water rather than for SIC, which will never concurrently be 100%.
“a worse behavior of step 1 would lead to a better validation metrics on step 3.”
The answer is no. If the U-net one predicts error segmented ice region, the precision metrics for U-net will be small, e.g., the OWrecall for U-net one will be 60%. Then Multi-textRG algorithm results an approximate error ice edges, similarly, the OWrecall for “SAR growing ice” will be also about 60% (because the algorithm is within the U-net ice region). Then the SAR-Chart SIC and SAR-AMSR SIC labels transmit this error. However, if U-net one performs very bad, the U-net two and three with totally same model structure and experiments settings will also performs bad, which also has nothing to do with labels. Thus, the design for U-net one is vital. The better-performing U-net one is designed and required to predict with concurrently OWrecall and ov40recall of near-100% (in our experiments, over 98%). The 99% OWrecall and 99% ov40recall validated from U-net two or three is not our goal, however, the 98% OWrecall and 98% ov40recall validated from U-net one is exactly our goal. In other words, we do not allow the totally failure of above step 1 existing. U-net one, two, and three are totally the same one model structure. The only difference is training labels, thus resulting different hyper-parameters.
The 99% OWrecall means 99% of the predicted-water is labelled-water, but not all labelled-water is correctly recognized. The 99% ov40recall means 99% of the predicted-over40% SIC is labelled-ice (>0% SIC), but not all labelled-ice is correctly recognized. Therefore, the bl40recall is always below 70% in Table 3, which means the labelled 0~40% SIC region is unstably predicted as ice (>0% SIC) or open water. Three recall metrics explain the predict precision together. The inter-comparison for metrics values of three U-net in Table 3 and 4 can reveal the ability of U-net model for detecting ice edges details changing with the different SIC labels, without needing other independent validation data. For example, we know the U-net one + Multi-textRG algorithm have produced more accurate ice edges compared to the over-drawn ice charts. The 99% OWrecall and 99% ov40recall from U-net two means the prediction is also more accurate, thus with more details, consistent with the “SAR growing ice”.
2) The “SAR growing ice”, resulted from U-net one + Multi-textRG algorithm, is validated based on the independent optical data---Landsat-8 and Sentinel-2 data. This is to evaluate the precision of the Multi-textRG algorithm for detailed ice detection. The results are given in Section 4.2 and 4.3.
For several reasons, we think it is not necessary to split the manuscript to two parts:
1. Again, the paper focus is always the Multi-textRG algorithm. “U-net one + Multi-textRG + optical data validation” is to demonstrate the chain of algorithm performs good. “U-net one” and “U-net three” is to demonstrate the Multi-textRG algorithm is important for producing high precision labels and thus for improving CNN model prediction.
2. We used the same U-net model developed and provided by the AI4Arctic project, without model innovation.
3. Training model for SIC prediction is not our goal. We only aim to solve the high precision ice and water classification. The precision limitation of SIC labels and the resolution limitation of SAR backscatter (compared to optical images) let the machine learning development for SIC prediction be much harder.
Thus, we think the “2. Sensitivity of U-net on level of details in input labels (U-net two, three, etc.)” is not enough to organize as another paper at present. And adding other model experiments for SIC prediction will be a high time cost. Thus, we insist on including the two parts in this manuscript because we hope the beautiful maps in Fig. 7 could give the reader valuable inspirations. Otherwise, we accept to reduce the second part from this current manuscript to highlight the Multi-textRG algorithm. We may make a decision after receiving more reviews.
We are not totally understand the concept that “blend in target labels into input data”. The input data for U-net one, two, and three are all Sentinel-1 HH and HV polarized images and AMSR2 36.5GHz H-polarization data. The training labels are different. “blend in target labels into input data”, does it mean that the “SAR-Chart label” is produced based on Sentinel-1 and AMSR2 data? In this way, the ice charts are also produced by manual interpretation on multi-source SAR, optical and radiometer data.
Thank you very much, Dr. Korosov, for your precious time. We really appreciate your reviews. And we except more discussions.
Citation: https://doi.org/10.5194/egusphere-2024-1177-AC1
-
AC1: 'Reply on RC1', Yan Sun, 06 Jun 2024
-
RC2: 'Comment on egusphere-2024-1177', Anonymous Referee #2, 26 Jun 2024
In this manuscript, titled “Combining the U-Net model and a Multi-textRG algorithm for fine SAR ice-water classification”, the authors have carried out extensive data analysis and machine learning model training and validation for sea ice classification. In my opinion, the major contribution of the work is the exploration of the combination of SAR (active) data and passive microwave imager (PMI) data for better mapping sea ice. The major source of information are from ice charts and AI4Arctic, Sentinel-1 images, PMI and derived data, as well as various products (Sentinel-2 and LandSat images). The use of data is extensive, involving a lot of work. The results are sound, although there are several issues to be dealt with. In my opinion, there are several shortcomings which are listed below as my major comments.
What puzzles me most is the main objective of the work, which apparently is multiple-purposed. If the main purpose is to distinguish ice from water at SAR-relevant scale, I would expect the use of SAR and PMI data sufficient, potentially trained against manually labeled SIC maps based on S2 and/or LandSat. The ice charts have several aspects to them, ice concentration being one, but arguably, other aspects such as the development stages (young, old, etc.) are more important for operations in icy waters. If the main purpose is to get better ice-water differentiation, why use ice chart data? Is it because of the practical issue, given the availability from AI4Arctic and related sources? This also relates to the validation as well, pls see my second comment.
Second, among the various payloads you have considered: SAR, ASI-SIC, AMSR-36GHz. The physical resolution is highest SAR. However, the merged product of ASI-SIC is also of relatively high resolution (3~5km). I think the authors should be very careful in claiming the superiority of the merged dataset, since there is no shown results for evaluating ASI-SIC product. The real advantage of merging may arise from the relatively high resolution of SAR, and the decent resolution, but complementary capability of PMI. Potentially, the real value is the combination of SAR and PMI. Some careful analysis may be added in this aspect.
Third, I consider a more serious use/protocol of the data is needed for revision. For example, the ice charts might not be independent from the SAR images as used for merging. The ice chart data, potentially produced by experts (or semi-automated systems assisted by experts) are treated as the standard for the training process. However, they may form the same data source for this study. At C-band, some classification problems do exist, especially regarding the relatively young ice with ambiguous backscatter properties with other types. Wind seas is always problematic due to high backscatter, which requires high-level texture information or PMI data to be differentiated. The physical limitations may cause uncertainty in ice charts as well, especially along the type boundaries (where the error is more likely present). I do not personally expect the authors to fully resolve this issues, but it is worthy to be aware of. Another example is the ASI-SIC data, which is mainly based on 89GHz data. The authors have carefully avoided using this data (due to atmospheric and oceanic influences), but one should be fully aware that this uncertainty is inherent in ASI-SIC product. The third example for data usage is the merging of the data should account for the different temporal representation of them. Note that both 36GHz and ASI-SIC of AMSR are based on daily mean PMI images, which may well consist of several satellite passes’ mean. SAR images are usually for a single pass, however. For thin ice/open water, the ice condition might have undergone significant changes, compromising the combination of these datasets. They may arise as key source of error when you approach the last several percentage of uncertainty. For the sake of completeness, they are definitely worthy to discuss formally.
Fourth, I find that the formulation and the writing is compromising the convey of the major idea of the paper. I suggest the authors: (1) be very clear of what you adopt from others and what your real innovation are; (2) formulate the discussion into a dedicated part (which is not included in the current form), not spread in the paper; (3) refrain from using too much new terms and focus on the key progress that has been made; (4) resort to English editing for better language use.
Citation: https://doi.org/10.5194/egusphere-2024-1177-RC2
Viewed
| HTML | XML | Total | Supplement | BibTeX | EndNote | |
|---|---|---|---|---|---|---|
| 188 | 63 | 13 | 264 | 81 | 5 | 5 |
- HTML: 188
- PDF: 63
- XML: 13
- Total: 264
- Supplement: 81
- BibTeX: 5
- EndNote: 5
Viewed (geographical distribution)
| Country | # | Views | % |
|---|
| Total: | 0 |
| HTML: | 0 |
| PDF: | 0 |
| XML: | 0 |
- 1