the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
| 07 Mar 2024
DeepPhenoMem V1.0: Deep learning modelling of canopy greenness dynamics accounting for multi-variate meteorological memory effects on vegetation phenology
Abstract. Vegetation phenology plays a key role in controlling the seasonality of ecosystem processes that modulate carbon, water and energy fluxes between biosphere and atmosphere. Accurate modelling of vegetation phenology in the interplay of Earth’s surface and the atmosphere is thus crucial to understand how the coupled system will respond to and shape climatic changes. Phenology is controlled by meteorological conditions at different time scales: on the one hand, changes in key meteorological variables (temperature, water, radiation) can have immediate effects on the vegetation development; on the other hand, phenological changes can be driven by past environmental conditions, known as memory effects. However, the processes governing meteorological memory effects on phenology are not completely understood, resulting in their limited performance of phenology simulated by land surface models. A deep learning model, specifically a long short-term memory network (LSTM), has the potential to capture and model the meteorological memory effects on vegetation phenology. Here, we apply the LSTM to model the vegetation phenology using meteorological drivers and canopy greenness at high temporal resolution collected taking advantage of digital repeat photography by the PhenoCam network. We compare a simple multiple linear regression model, a no-memory-effect, and a full-memory-effect LSTM model to predict the whole seasonal greenness trajectory and the corresponding phenological transition dates of 50 sites and 317 site-year during 2009–2018, across deciduous broadleaf forests, evergreen needleleaf forests and grasslands. The deep learning model outperforms the multiple linear regression model, and the full-memory-effect LSTM model performs better than no-memory-effect model for all three plant function types (median R2 of 0.878, 0.957, and 0.955 for broadleaf forests, evergreen needleleaf forests and grasslands) corroborating the benefits of deep learning approach and the importance of multi-variate meteorological memory effects in phenology modelling. We also find that the LSTM model is capable of predicting the seasonal dynamic variations of canopy greenness and reproducing trends in shifting phenological transition dates. We also performed a sensitivity analysis of the LSTM model to assess its plausibility, revealing its coherence with established knowledge of vegetation phenology sensitivity to meteorological conditions, particularly changes in temperature. Our study highlights that 1) multi-variate meteorological memory effects play a crucial role in vegetation phenology, and 2) deep learning opens up new avenues for improving the representation of vegetation phenological processes in land surface models via a hybrid modelling approach.
-
Notice on discussion status
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
-
Preprint
(2972 KB)
-
Supplement
(917 KB)
-
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
- Preprint
(2972 KB) - Metadata XML
-
Supplement
(917 KB) - BibTeX
- EndNote
- Final revised paper
Journal article(s) based on this preprint
Interactive discussion
Status: closed
-
CC1: 'Comment on egusphere-2024-464', Matthew Garcia, 11 Mar 2024
line 79: Ren et al. (2021) is not listed in the References.
line 378: reference information for Hänninen (1990) is incomplete.
line 384 and 395: to which of the two Liu et al. (2018) references does each of these refer?
lines 450-451: "These results underscore the capability of our deep learning framework to retrieve fundamental physical information solely from data." This is not shown or proven in this paper. Note lines 399-400: "... the specific contributions of different meteorological factors to memory effects on vegetation development remain unclear." The fact that the model produces phenological shifts due to warming that are consistent with other studies does not necessarily show that you're getting the right answer for the right reasons. You're getting a good answer that is generally right, but until you can pick apart the NN nodes in order to show that they have developed causal transformations that are consistent with physical observations of vegetation responses to the various input variables, you cannot claim "meaningful physical insights" (line 448). You have developed a simulation of the outcome of the phenological process, not a dynamical or mechanistic model of the process itself.
Citation: https://doi.org/10.5194/egusphere-2024-464-CC1 -
AC1: 'Reply on CC1', Guohua Liu, 13 Mar 2024
Thank you very much for your interests in our study and providing comments!
- We will revise the references as you suggested.
- We acknowledge your point regarding the need for more robust evidence to support our assertion about the extraction of fundamental physical information solely from data. While we agree that our study does not definitively prove that the deep learning model did catch the underlying biological process, it is clear that it is able to reproduce the sensitivity of the phenology to temperature. We understand that the claims in our paper might be misunderstood as a stronger claim and will adjust our language accordingly to ensure that our claims are appropriately grounded.
Once again, we appreciate your valuable input, and we thank you for helping us improve the clarity and accuracy of our work.
Citation: https://doi.org/10.5194/egusphere-2024-464-AC1
-
AC1: 'Reply on CC1', Guohua Liu, 13 Mar 2024
-
CEC1: 'Comment on egusphere-2024-464', Juan Antonio Añel, 28 Mar 2024
Dear authors,
I would like to bring to your attention an issue with your manuscript. You have deposited the code (Python routines) that you use in your work in a Zenodo repository. However, neither the Python version nor the version of the relevant libraries used (Pandas, Sklearn, Torch, etc.) is mentioned. The implementation of different algorithms can change with different versions, or even be non-existent in some of them. The lack of this information makes difficult to replicate and reproduce your work. Therefore, please, reply to this comment with the version numbers for the software that you have used, and include them in your manuscript in case that you submit a reviewed version or it is accepted.
Juan A. Añel
Geosci. Model Dev. Executive Editor
Citation: https://doi.org/10.5194/egusphere-2024-464-CEC1 -
AC2: 'Reply on CEC1', Guohua Liu, 04 Apr 2024
Dear Juan,
Thank you for bringing to our attention the oversight regarding the versions of Python and relevant libraries used in our study. We sincerely apologize for any inconvenience this may have caused and greatly appreciate your commitment to upholding the standards of quality and reproducibility.
Please find below the version numbers for the software and libraries utilized in our study:
- Python: 3.9.18
- Pandas: 1.5.3
- PyTorch: 1.12.1
- Numpy: 1.26.4
We will promptly update the code repository documentation with these details to ensure clarity and transparency of our work.
Thank you once again for bringing this matter to our attention.
Sincerely,
Guohua Liu
On behalf of all authors
Citation: https://doi.org/10.5194/egusphere-2024-464-AC2
-
AC2: 'Reply on CEC1', Guohua Liu, 04 Apr 2024
-
RC1: 'Comment on egusphere-2024-464', Anonymous Referee #1, 06 Apr 2024
Review of “DeepPhenoMem V1.0: Deep learning modelling of canopy greenness
dynamics accounting for multi-variate meteorological memory effects on vegetation phenology” by Liu et al. for consideration in EGUsphere.
Based on in situ observation data of plant phenology at 50 sites distributed across Northern American, the authors trained a deep learning model LSTM which has the potential to capture and model the meteorological memory effects on vegetation phenology. The topic of this study is very important and interesting. It provides a new pathway to simulate and investigate the complex impacts of environmental factors on plant canopy dynamics. In addition, the manuscript is overall well organized, and the methods used in this study have been introduced detailedly.
Nonetheless, I still have some questions on the method and results of this study:
- The DeepPhenoMem model in this study is trained at 45 sites, and evaluated at 5 sites. Some random factors (e.g. the specific climate and species in the 4 validation sites compared to the training sites) might strongly affect the evaluation results. To give a more robust evaluation on the model, I would suggest the authors to do a 10-fold (or 5-fold) cross validation.
- There are 3 unseen sites for deciduous broadleaved forests were used to test the trained model in this study. Why not show the evaluation results at all of these 3 sites in Figs. 5 & 6. The readers might wonder that only the site with best model performance for PFT DB was showed in Figs. 5 & 6. The evaluation results might thus be biased.
- The deciduous broadleaved forests (DB) generally show stronger seasonal variability compared to evergreen needle-leaved (EN) forests. I am wondering why the DeepPhenoMem model performs better for EN, compared to DB (Table 1, Fig. 3). If it is only because the BD has been tested at three sites, while the EN was only tested in 1 sites? In theory, the observed and simulated GCC for DB and grassland could be low to 0. Why the GCC for DB and grassland are all higher than 0.3. Are there any evergreen plants living in the DB and grassland sites. In addition, the EN keeps being green across the year, is it accurate enough to extracted the GCC from the digital images photographed by automated and high-frequency digital cameras. Even in the end of growing season, the GCC for EN should still be high.
- Based on the observed data, the authors can actually calculate the sensitivities of SOS and EOS to warming using linear regression (e.g. Fu et al., 2015, Nature). I am wondering if the temperature sensitivities simulated from the trained DeepPhenoMem model in this study are comparable to the values calculated based on observations. I would suggest the authors to do a comparison/evaluation.
Specific comments:
L147-148: Is it accurate enough to use the sum of precipitation over the previous month as a proxy of daily SW? Is there any reference of Eq. 3. Maybe it is better to simply say the sum of precipitation over the previous month has been included in the model, without mentioning the SW.
L163-164: Why not conduct a cross-validation?
Table 1: Please add RMSE of each model for each vegetation type
Fig. 4: What does the rhombus represent here? The significance of difference between two versions of the model? If no, please provide a significance test between results from the M0 and Mfull. The sub-plots a, b, c show results for DB, EN, GR, respectively?
Figs. 5 & 6: There are 3 unseen sites for DB (Fig. 1), why only results for harvardbarn2 was presented?
L376-378: I did not find what result in this study indicate the cumulative thermal summation, rather than daily temperature alone, determines vegetation phenology
L406-411: Not fully true. The phenology module of Earth System models (ESMs) indeed only focuses on a few specific phenological events (e.g. start and end of the growing season). However, the ESMs also simulate the whole time series of canopy development/evolution across the whole growing season, by mechanically simulating the photosynthesis, autotrophic respiration, carbon allocation, etc. To have a closed mass balance of carbon, the canopy evolution has to be simulated mechanically, rather than using an empirical model or machine learning model.
Citation: https://doi.org/10.5194/egusphere-2024-464-RC1 - AC3: 'Reply on RC1', Guohua Liu, 10 May 2024
-
RC2: 'Comment on egusphere-2024-464', Anonymous Referee #2, 19 Apr 2024
Liu et al.,
DeepPhenoMem V1.0: Deep learning modelling of canopy greenness dynamics accounting for multi-variate meteorological memory effects on vegetation phenology
General comments:
This study aims to train an LTSM model to simulate the temporal evolution of a measure of canopy greenness observed with in-situ repeat, digital photography across three plant functional types, testing performance against the final year of observations at the ~50 training sites, and across multiple years at several sites not used in training. This is an interesting proposal and, given the current limitations in long established phenology models, potentially very useful to a wide community of vegetation modelers. The primary conclusions that the LTSM model seems to capture some of the underlaying controls on phenology, that incorporating meteorological memory effects improves performance, and the models exhibits plausible relationships are sound.
However, the results as presented are difficult to interpret beyond this – the reliance on R2 statistics is potentially misleading considering the marked biases exhibited by the model. Incorporating RMSE or some other measure of bias throughout the results and discussion is required to better understand where and when the model performs well or otherwise. Contrasting and explaining (lack of) model performance across space/time/PFT would potentially be more informative than the current approach which tends towards endorsing model performance in very vague terms. At a minimum, this would include adding RMSE values to Table 1 and including in Figure 4 and moving Figure 9 from the Discussion to the Results and maybe incorporating more discussion of Figure S2, as well as adding additional interpretation and discussion of these results related to the bias.
Specific comments:
L126: Need some additional information that explains the limited and very similar dynamical range in GCC across the three PFTs. Why is there not more annual variation in deciduous v. evergreen trees in particular?
L153: It doesn’t seem appropriate to call weighted mean monthly precipitation “soil water” – it isn’t and should be renamed. Also, it was an unfortunate choice of variable to include when trying to tease out the difference between models with and without memory effects as by design it will capture an approximation of soil moisture memory that won’t be removed by the shuffling in the M0 model and leads to a mixture of instantaneous and time integrating variables in the regression model. If this can be included, then why not some cumulative temperature term, or day of year etc. which are known to strongly influence phenology.
L251: Figures 1, 4 and Figure S1 indicate there are three deciduous test sites? It’s unclear which (maybe all?) are being shown in Figure 3
L270: Here, and elsewhere in several places in the manuscript, reference to figure numbers is incorrect. Please check all these carefully.
L276: There is evidently zero/minimal skill in simulating daily anomalies, whilst overall R2 values indicate some skill in seasonal variability/monthly times scales. Is attempting to simulate what are presumably noisy daily data a valuable test? Can some smoothing be applied to investigate if there is any model skill between daily and monthly time scales. Or is there only really skill in seasonal variability - making additional analysis of variation of a few days in SOS and EOS difficult to interpret?
L280: Here is a clear example of the bias that needs to be quantified and examined more thoroughly throughout the whole analysis.
Citation: https://doi.org/10.5194/egusphere-2024-464-RC2 - AC4: 'Reply on RC2', Guohua Liu, 10 May 2024
Interactive discussion
Status: closed
-
CC1: 'Comment on egusphere-2024-464', Matthew Garcia, 11 Mar 2024
line 79: Ren et al. (2021) is not listed in the References.
line 378: reference information for Hänninen (1990) is incomplete.
line 384 and 395: to which of the two Liu et al. (2018) references does each of these refer?
lines 450-451: "These results underscore the capability of our deep learning framework to retrieve fundamental physical information solely from data." This is not shown or proven in this paper. Note lines 399-400: "... the specific contributions of different meteorological factors to memory effects on vegetation development remain unclear." The fact that the model produces phenological shifts due to warming that are consistent with other studies does not necessarily show that you're getting the right answer for the right reasons. You're getting a good answer that is generally right, but until you can pick apart the NN nodes in order to show that they have developed causal transformations that are consistent with physical observations of vegetation responses to the various input variables, you cannot claim "meaningful physical insights" (line 448). You have developed a simulation of the outcome of the phenological process, not a dynamical or mechanistic model of the process itself.
Citation: https://doi.org/10.5194/egusphere-2024-464-CC1 -
AC1: 'Reply on CC1', Guohua Liu, 13 Mar 2024
Thank you very much for your interests in our study and providing comments!
- We will revise the references as you suggested.
- We acknowledge your point regarding the need for more robust evidence to support our assertion about the extraction of fundamental physical information solely from data. While we agree that our study does not definitively prove that the deep learning model did catch the underlying biological process, it is clear that it is able to reproduce the sensitivity of the phenology to temperature. We understand that the claims in our paper might be misunderstood as a stronger claim and will adjust our language accordingly to ensure that our claims are appropriately grounded.
Once again, we appreciate your valuable input, and we thank you for helping us improve the clarity and accuracy of our work.
Citation: https://doi.org/10.5194/egusphere-2024-464-AC1
-
AC1: 'Reply on CC1', Guohua Liu, 13 Mar 2024
-
CEC1: 'Comment on egusphere-2024-464', Juan Antonio Añel, 28 Mar 2024
Dear authors,
I would like to bring to your attention an issue with your manuscript. You have deposited the code (Python routines) that you use in your work in a Zenodo repository. However, neither the Python version nor the version of the relevant libraries used (Pandas, Sklearn, Torch, etc.) is mentioned. The implementation of different algorithms can change with different versions, or even be non-existent in some of them. The lack of this information makes difficult to replicate and reproduce your work. Therefore, please, reply to this comment with the version numbers for the software that you have used, and include them in your manuscript in case that you submit a reviewed version or it is accepted.
Juan A. Añel
Geosci. Model Dev. Executive Editor
Citation: https://doi.org/10.5194/egusphere-2024-464-CEC1 -
AC2: 'Reply on CEC1', Guohua Liu, 04 Apr 2024
Dear Juan,
Thank you for bringing to our attention the oversight regarding the versions of Python and relevant libraries used in our study. We sincerely apologize for any inconvenience this may have caused and greatly appreciate your commitment to upholding the standards of quality and reproducibility.
Please find below the version numbers for the software and libraries utilized in our study:
- Python: 3.9.18
- Pandas: 1.5.3
- PyTorch: 1.12.1
- Numpy: 1.26.4
We will promptly update the code repository documentation with these details to ensure clarity and transparency of our work.
Thank you once again for bringing this matter to our attention.
Sincerely,
Guohua Liu
On behalf of all authors
Citation: https://doi.org/10.5194/egusphere-2024-464-AC2
-
AC2: 'Reply on CEC1', Guohua Liu, 04 Apr 2024
-
RC1: 'Comment on egusphere-2024-464', Anonymous Referee #1, 06 Apr 2024
Review of “DeepPhenoMem V1.0: Deep learning modelling of canopy greenness
dynamics accounting for multi-variate meteorological memory effects on vegetation phenology” by Liu et al. for consideration in EGUsphere.
Based on in situ observation data of plant phenology at 50 sites distributed across Northern American, the authors trained a deep learning model LSTM which has the potential to capture and model the meteorological memory effects on vegetation phenology. The topic of this study is very important and interesting. It provides a new pathway to simulate and investigate the complex impacts of environmental factors on plant canopy dynamics. In addition, the manuscript is overall well organized, and the methods used in this study have been introduced detailedly.
Nonetheless, I still have some questions on the method and results of this study:
- The DeepPhenoMem model in this study is trained at 45 sites, and evaluated at 5 sites. Some random factors (e.g. the specific climate and species in the 4 validation sites compared to the training sites) might strongly affect the evaluation results. To give a more robust evaluation on the model, I would suggest the authors to do a 10-fold (or 5-fold) cross validation.
- There are 3 unseen sites for deciduous broadleaved forests were used to test the trained model in this study. Why not show the evaluation results at all of these 3 sites in Figs. 5 & 6. The readers might wonder that only the site with best model performance for PFT DB was showed in Figs. 5 & 6. The evaluation results might thus be biased.
- The deciduous broadleaved forests (DB) generally show stronger seasonal variability compared to evergreen needle-leaved (EN) forests. I am wondering why the DeepPhenoMem model performs better for EN, compared to DB (Table 1, Fig. 3). If it is only because the BD has been tested at three sites, while the EN was only tested in 1 sites? In theory, the observed and simulated GCC for DB and grassland could be low to 0. Why the GCC for DB and grassland are all higher than 0.3. Are there any evergreen plants living in the DB and grassland sites. In addition, the EN keeps being green across the year, is it accurate enough to extracted the GCC from the digital images photographed by automated and high-frequency digital cameras. Even in the end of growing season, the GCC for EN should still be high.
- Based on the observed data, the authors can actually calculate the sensitivities of SOS and EOS to warming using linear regression (e.g. Fu et al., 2015, Nature). I am wondering if the temperature sensitivities simulated from the trained DeepPhenoMem model in this study are comparable to the values calculated based on observations. I would suggest the authors to do a comparison/evaluation.
Specific comments:
L147-148: Is it accurate enough to use the sum of precipitation over the previous month as a proxy of daily SW? Is there any reference of Eq. 3. Maybe it is better to simply say the sum of precipitation over the previous month has been included in the model, without mentioning the SW.
L163-164: Why not conduct a cross-validation?
Table 1: Please add RMSE of each model for each vegetation type
Fig. 4: What does the rhombus represent here? The significance of difference between two versions of the model? If no, please provide a significance test between results from the M0 and Mfull. The sub-plots a, b, c show results for DB, EN, GR, respectively?
Figs. 5 & 6: There are 3 unseen sites for DB (Fig. 1), why only results for harvardbarn2 was presented?
L376-378: I did not find what result in this study indicate the cumulative thermal summation, rather than daily temperature alone, determines vegetation phenology
L406-411: Not fully true. The phenology module of Earth System models (ESMs) indeed only focuses on a few specific phenological events (e.g. start and end of the growing season). However, the ESMs also simulate the whole time series of canopy development/evolution across the whole growing season, by mechanically simulating the photosynthesis, autotrophic respiration, carbon allocation, etc. To have a closed mass balance of carbon, the canopy evolution has to be simulated mechanically, rather than using an empirical model or machine learning model.
Citation: https://doi.org/10.5194/egusphere-2024-464-RC1 - AC3: 'Reply on RC1', Guohua Liu, 10 May 2024
-
RC2: 'Comment on egusphere-2024-464', Anonymous Referee #2, 19 Apr 2024
Liu et al.,
DeepPhenoMem V1.0: Deep learning modelling of canopy greenness dynamics accounting for multi-variate meteorological memory effects on vegetation phenology
General comments:
This study aims to train an LTSM model to simulate the temporal evolution of a measure of canopy greenness observed with in-situ repeat, digital photography across three plant functional types, testing performance against the final year of observations at the ~50 training sites, and across multiple years at several sites not used in training. This is an interesting proposal and, given the current limitations in long established phenology models, potentially very useful to a wide community of vegetation modelers. The primary conclusions that the LTSM model seems to capture some of the underlaying controls on phenology, that incorporating meteorological memory effects improves performance, and the models exhibits plausible relationships are sound.
However, the results as presented are difficult to interpret beyond this – the reliance on R2 statistics is potentially misleading considering the marked biases exhibited by the model. Incorporating RMSE or some other measure of bias throughout the results and discussion is required to better understand where and when the model performs well or otherwise. Contrasting and explaining (lack of) model performance across space/time/PFT would potentially be more informative than the current approach which tends towards endorsing model performance in very vague terms. At a minimum, this would include adding RMSE values to Table 1 and including in Figure 4 and moving Figure 9 from the Discussion to the Results and maybe incorporating more discussion of Figure S2, as well as adding additional interpretation and discussion of these results related to the bias.
Specific comments:
L126: Need some additional information that explains the limited and very similar dynamical range in GCC across the three PFTs. Why is there not more annual variation in deciduous v. evergreen trees in particular?
L153: It doesn’t seem appropriate to call weighted mean monthly precipitation “soil water” – it isn’t and should be renamed. Also, it was an unfortunate choice of variable to include when trying to tease out the difference between models with and without memory effects as by design it will capture an approximation of soil moisture memory that won’t be removed by the shuffling in the M0 model and leads to a mixture of instantaneous and time integrating variables in the regression model. If this can be included, then why not some cumulative temperature term, or day of year etc. which are known to strongly influence phenology.
L251: Figures 1, 4 and Figure S1 indicate there are three deciduous test sites? It’s unclear which (maybe all?) are being shown in Figure 3
L270: Here, and elsewhere in several places in the manuscript, reference to figure numbers is incorrect. Please check all these carefully.
L276: There is evidently zero/minimal skill in simulating daily anomalies, whilst overall R2 values indicate some skill in seasonal variability/monthly times scales. Is attempting to simulate what are presumably noisy daily data a valuable test? Can some smoothing be applied to investigate if there is any model skill between daily and monthly time scales. Or is there only really skill in seasonal variability - making additional analysis of variation of a few days in SOS and EOS difficult to interpret?
L280: Here is a clear example of the bias that needs to be quantified and examined more thoroughly throughout the whole analysis.
Citation: https://doi.org/10.5194/egusphere-2024-464-RC2 - AC4: 'Reply on RC2', Guohua Liu, 10 May 2024
Peer review completion
Journal article(s) based on this preprint
Viewed
| HTML | XML | Total | Supplement | BibTeX | EndNote | |
|---|---|---|---|---|---|---|
| 620 | 199 | 45 | 864 | 58 | 24 | 24 |
- HTML: 620
- PDF: 199
- XML: 45
- Total: 864
- Supplement: 58
- BibTeX: 24
- EndNote: 24
Viewed (geographical distribution)
| Country | # | Views | % |
|---|
| Total: | 0 |
| HTML: | 0 |
| PDF: | 0 |
| XML: | 0 |
- 1
Guohua Liu
Mirco Migliavacca
Christian Reimers
Basil Kraft
Markus Reichstein
Andrew Richardson
Lisa Wingate
Nicolas Delpierre
Alexander Winkler
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
- Preprint
(2972 KB) - Metadata XML
-
Supplement
(917 KB) - BibTeX
- EndNote
- Final revised paper