the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
Uncertainty Assessment of Satellite Remote Sensing-based Evapotranspiration Estimates: A Systematic Review of Methods and Gaps
Johannes van der Kwast
Solomon Seyoum
Remko Uijlenhoet
Graham Jewitt
Marloes Mul
Abstract. Satellite remote sensing (RS) data are increasingly being used to estimate total evaporation or evapotranspiration (ET) over large regions. Since RS-based ET (RS-ET) estimation inherits uncertainties from several sources, many available studies have assessed these uncertainties using different methods and reference data. However, the suitability of methods and reference data subsequently affects the validity of these evaluations. This study summarizes the status of the various methods applied for uncertainty assessment of RS-ET estimates, discusses the advances and caveats of these methods, identifies assessment gaps, and provides recommendations for future studies. We systematically reviewed 601 research papers published from 2011 to 2021 that assessed the uncertainty or accuracy of RS-ET estimates. We categorized and classified them based on (i) the methods used to assess uncertainties, (ii) the context where uncertainties were evaluated, and (iii) the metrics used to report uncertainties. Our quantitative synthesis shows that the uncertainty assessments of RS-ET estimates are not consistent and comparable in terms of methodology, reference data, geographical distribution, and uncertainty presentation. Most studies used validation methods using Eddy Covariance (EC) based ET estimates as reference. However, in many regions such as Africa and the Middle East, other references are often used due to the lack of EC stations. The accuracy and uncertainty of RS-ET estimates are most often described by Root-Mean-Squared Error (RMSE). When validating against EC-based estimates, the RMSE of daily RS-ET varies greatly among different locations and levels of temporal support, ranging from 0.01 to 6.65 mm/day with a mean of 1.12 mm/day. We conclude that future studies need to report the context of validation, the uncertainty of the reference datasets, the mismatch in temporal and spatial scales of reference datasets to that of the RS-ET estimates, and multiple performance metrics with their variation in different conditions and statistical significance to provide a comprehensive interpretation to assist potential users. We provide specific recommendations in this regard. Furthermore, extending the application of RS-ET to regions that lack validation will require obtaining additional ground-based data and combining different methods for uncertainty assessment.
- Preprint
(1598 KB) -
Supplement
(488 KB) - BibTeX
- EndNote
Bich Ngoc Tran et al.
Status: open (until 20 Jun 2023)
-
RC1: 'Comment on egusphere-2023-725', Joshua Fisher, 17 May 2023
reply
The comment was uploaded in the form of a supplement: https://egusphere.copernicus.org/preprints/2023/egusphere-2023-725/egusphere-2023-725-RC1-supplement.pdf
-
RC2: 'Comment on egusphere-2023-725', Anonymous Referee #2, 22 May 2023
reply
First of all, I would like to extend my congratulations to the authors for their valuable research presented in the article titled "Uncertainty Assessment of Satellite Remote Sensing-based Evapotranspiration Estimates: A Systematic Review of Methods and Gaps." The authors have demonstrated a significant research effort and I acknowledge the extensive work invested in this study. However, I believe it would be beneficial for the ET community if the discussion also focused on the performance and uncertainties of the analysed products/models, as well as the underlying reasons for their uncertainty/performance. In light of this, I have several comments and suggestions that I believe can contribute to enhancing the manuscript.
Â
1. Due to the nature of a systematic review, it is difficult to differentiate between articles that evaluate the performance of existing ET products and ET-based models. It would be very beneficial to clarify the distinction between evaporation products and the models used to estimate ET. Currently, it is challenging for readers to differentiate between them, making it difficult to follow certain ideas. For instance, in Line 231, the authors discuss eight topics for assessing uncertainty in RS-ET, where some points relate to the evaluation of ET products while others to the models. It would be beneficial to clearly indicate what is defined as RS-ET inthe manuscript and which results are from models or open-acces gridded products.
Â
2. The article is lengthy, and it would be beneficial to condense the sections "Theoretical frameworks" and "Systematic quantitative literature review method" for brevity.
Â
3. The manuscript could benefit from discussing which methods and products perform better in specific contexts. It would be helpful to provide insights on the performance of models and products in relation to specific regions, climates, and relevant factors. For example, i) identifying the errors associated with each method/product; ii) the reported advantages and disadvantages of different models/products; iii) important parameters that drive the estimation of ET in existing models; iv) lessons learned from previous evaluations; and v) which models/products have demonstrated higher physical consistency.
Â
4. In the section "Review of methods for RS-ET uncertainty assessment", the authors could focus on the performance of models/products and relate their findings to specific regions and climates when reported. Addressing questions such as which models performed better in certain areas and why, the sources of uncertainty, the relevance of spatio-temporal resolution in operational applications, the impact of geographical features on model/product uncertainty, and the influence of climate on product performance would greatly enhance this section.
Â
5. Consider reducing the use of acronyms that are infrequently mentioned in the manuscript, as it can improve readability and comprehension.
Â
6. The authors should clarify the timeframe of their study. While they mention focusing on the period from 2011, the end date or year is only specified on L187 stating that the databases were last accessed on 21.09.2021. It would be very valuable to update the research up to a more recent date to provide a comprehensive evaluation :)
Â
Â
7. The authors mention using keywords like "accuracy," "bias," and "precision" to assess uncertainty in products, although these terms differ from the proper definition of uncertainty. It would be important to include the term "performance" in the evaluation, as many studies summarize their findings in terms of model or product performance.
Â
8. Section 6, "Results of RS-ET uncertainty assessment," primarily evaluates articles based on RMSE. However, comparing articles solely on RMSE is not very meaningful, as this goodnes-of-fit metric does not allow for comparisons across areas with different climates and ET patterns. Therefore, the metrics presented in Table 4 (median, mean, quartiles, standard deviations) and Figure 14, which are grouped by evaluated temporal scale may be misleading. A good and valuable reccomentation that the athors could use in their artile could be related to the fact that researchers should report uncertainty/performance metrics using indices that are comparable across studies and not influenced by regional climate or specific ET patterns. It would be valuable to discuss about which metric is reported to be better proxy of model/product performance and which models/products performed better.
Â
9. The summary of the manuscript could address relevant questions for researchers and practitioners, such as recommended evaluations for assessing the performance of ET products when data is and is not available.
___________________________________________________________________________________________________________________
Furthermore, I would like to provide a few additional minor suggestions for improvement:
Â
L10: The authors can emphasise here that evapotranspiration is often referred to as evaporation. As it is currently written, it seems that the authors are referring to both evaporation and evapotranspiration.
Â
L39-42: Here, the authors mention some methods, but the list is not exhaustive. They can add GLEAM to this list for example, which is a well-known method that drives a ET product with the same name.
Â
L44-45: This sentence can be rewritten for better clarity.
Â
L46: The authors mention that retrieving ET estimates from some models requires expertise about the models. However, this is true for every model, so this sentence can be deleted.
Â
L50: This sentence is a bit convoluted. It would be helpful if the authors could clarify their intended meaning.
Â
L51-53: Here, the authors mention that uncertainty assessment helps data users determine the level of confidence they can have in ET estimates and inferred information about water resources. Since readers of this article may be researchers exploring ET products and models for the first time, it would be a good idea to mention that the use of the products is also limited by their spatio-temporal resolution, specific applications, and latency.
Â
L55: "foci" should be changed to "focus." The focus of multiple articles is explained in Table S2.
Â
L59: What do the authors mean by "spatial data production"?
Â
L60: What do the authors mean by "a good practice protocol for operational validation"?
Â
L59: What do the authors mean by "complete documentation"?
Â
Figure 1: This figure is very good and helpful. It will surely assist readers in accessing previous literature review articles. Could the authors complete the list of existing manuscripts related to the review of RS-ET estimation, uncertainty, and validation of products (and models)?
Â
L130: "reanalyzes" should be "reanalyses." Additionally, could the authors rewrite this sentence to better explain what is considered a high level of processing?
Â
L143: What about replacing "true" with "more accurately representing the ET values"?
Â
Figure 3: Please replace "support" with "resolution." Why does the model calculation not have a number? There is uncertainty regarding whether the model is able to resemble physical processes or not. Finally, the authors can mention in the figure that compound uncertainty is the sum of all other uncertainties.
Â
L153: Why specifically refer to Monte Carlo when there are more advanced techniques to assess uncertainty propagation?
Â
L165: Can the authors add a sentence on how the definition of validation has changed over time?
Â
L170: This sentence is not very clear to me. What do the authors mean by "model validation and data" in this context? Maybe the parentheses are misplaced and disrupt the flow of the sentence?
Â
L171-172: Can this sentence be deleted? I think the idea is clearly explained in the following sentences.
Â
L176: This sentence could be rewritten for clarity. Something like: "Validating a model used to derive ET estimates does not necessarily imply that it can be used with different forcing data and provide accurate results. Therefore, when a model is applied to derive ET estimates with different forcings or in different settings, its performance must be evaluated." In the current version, it is difficult to disentangle what is a model, an ET product, and a product based on running the model with different forcings :)
Â
L183: "by" instead of "tby"
Â
L189-190 and Table 1: It would be interesting to know how these terms were chosen. What about other terms like "performance," "quality" (alone), and "error"?
Â
L200: "process" instead of "system."
Â
Figure 4: What does "not using the same method to report uncertainty" mean?
Â
Figure 5 and 9: Why are there 38 articles without any link to a topic? Could the authors provide an explanation in the caption?
Â
Figure 6: It is difficult to see the low values on the graph. Maybe consider using a barplot to visualise this more straightforwardly.
Â
L256: "Estimate ET" instead of "observe ET." Remember that ET cannot be directly observed ;)
Â
Â
Figure 7: I really liked this figure! In the caption, the authors can add an explanation of the "others" category. Are irrigation and water balance articles combined in this category?
Â
L303: Here, the authors could briefly mention the assumptions of the simplified water balance.
Â
L309: Still less known compared to what? Maybe rephrase the sentence to clarify.
Â
L342: Some acronyms are introduced more than once, e.g., SA.
Â
L4447: Maybe consider renaming this subsection to something other than "Research Objectives." For example: "Assessment based on the objectives of the analysed manuscripts."
Â
L580: There is a missing space between "in" and "a."
Â
L580-581: I completely agree that further research should combine local and global evaluation efforts, but including a reason for this in the text could be very beneficial for the readers.
Â
L593-593: I do not completely agree with this argument. The RMSE ranges could serve as a baseline, but we have to keep in mind that they are not directly comparable.
Â
L601: What do the authors mean by "matched as much as possible"?
Â
L602: I do not completely agree with this statement. We should report metrics that enable a fair comparison between regions with different climates/patterns.
Â
L605: What do the authors mean by this statement? Please provide further clarification.
Â
L611-613: I did not understand this sentence. Could the authors please provide additional clarification or rephrase the sentence for clarity?
Â
I hope these comments and suggestions are helpful in improving the manuscript. Once again, congratulations to the authors on their research, and I look forward to reading a revised version of the manuscript.
Â
Citation: https://doi.org/10.5194/egusphere-2023-725-RC2 -
RC3: 'Comment on egusphere-2023-725', Anonymous Referee #3, 28 May 2023
reply
‘Uncertainty Assessment of Satellite Remote Sensing-based Evapotranspiration Estimates: A Systematic Review of Methods and Gaps’ by Tran et al., HESS-2023-725
The manuscript surveyed and reviewed the status of the various methods used for uncertainty assessment of remote sensing based estimation of evapotranspiration. It discussed the advances and caveats of the different methods, identified assessment gaps, and provided recommendations for future studies.
This reviewer considers such an assessment very useful for the community in using the various RS-ET estimates in hydrological studies. It feels however that some important aspects are missing which concern the model physics and dynamics and the considered physical processes in estimating ET using remote sensing data as input. The urgent challenge to the hydrological remote sensing community is therefore investigating the physics and dynamics of the processes involved in evapotranspiration and devising adequate methods to represent such processes in generating the RS-ET estimates. Once a chosen model is able to adequately represent such physics and dynamics for a few quality controlled reference in-situ sites, the uncertainty in their application to other sites and the globe is considerably reduced, because we can confidently expect that the physics is the same everywhere and the dynamics can be attributed to the temporal resolution of the model and the input data. Â
Fig. 14 needs some more explanation for the different symbols (this is obviously a box plot, but it is not clear to the reader by itself what the different statistics are compared to Table 4).
Citation: https://doi.org/10.5194/egusphere-2023-725-RC3
Bich Ngoc Tran et al.
Data sets
Systematic Quantitative Literature Review - Uncertainty assessment of Evapotranspiration Remote Sensing Bich Tran https://doi.org/10.4121/797dcaff-56e3-45ae-a931-f6f4a3135d26.v1
Meta-analysis of Remotely sensed Evapotranspiration validation with Eddy Covariance Bich Tran and Marloes Mul https://doi.org/10.4121/e6e1713a-0c2b-4775-a7f4-9e6e0b2cf40f.v1
Bich Ngoc Tran et al.
Viewed
HTML | XML | Total | Supplement | BibTeX | EndNote | |
---|---|---|---|---|---|---|
347 | 164 | 12 | 523 | 26 | 2 | 2 |
- HTML: 347
- PDF: 164
- XML: 12
- Total: 523
- Supplement: 26
- BibTeX: 2
- EndNote: 2
Viewed (geographical distribution)
Country | # | Views | % |
---|
Total: | 0 |
HTML: | 0 |
PDF: | 0 |
XML: | 0 |
- 1