the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
| 28 Feb 2024
Evaluation of CMIP6 Models Performance in Simulating Historical Biogeochemistry across Southern South China Sea
Abstract. This study evaluates the ability of Earth System Models (ESMs) from the Coupled Model Intercomparison Project Phase 6 (CMIP6) to simulate biogeochemical variables in the southern South China Sea (SCS). The analysis focuses on chlorophyll, phytoplankton, nitrate, and oxygen at annual and seasonal scales. The models' performance is assessed against Copernicus Marine Environment Monitoring Service (CMEMS) data using statistical metrics such as the Taylor diagram and Taylor skill score. The results show that the models generally capture the observed spatial patterns of biogeochemical variables but exhibit varying degrees of overestimation or underestimation. The performance of the models is also influenced by the season, with some models showing better performance during the southwest monsoon than the northeast monsoon. Overall, the top five best-performing models for biogeochemical variables are MIROC-ES2H, GFDL-ESM4, CanESM5-CanOE, MPI-ESM1-2-LR, and NorESM2-LM. The findings of this study have implications for researchers and end-users of the datasets, providing guidance for model improvement and understanding the impacts of climate change on the SCS ecosystem.
-
Notice on discussion status
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
-
Preprint
(1989 KB)
-
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
- Preprint
(1989 KB) - Metadata XML
- BibTeX
- EndNote
- Final revised paper
Journal article(s) based on this preprint
Interactive discussion
Status: closed
-
RC1: 'Comment on egusphere-2024-72', Anonymous Referee #1, 23 Apr 2024
In this manuscript, Marshal et al. scrutinize the performance of CMIP6 (state-of-the-art) Earth system models to simulate biogeochemical variables in the southern South China Sea (SCS).
.
The manuscript is clearly written and proposes a sound methodology. The results are well explained and discussed through the manuscript. This work presents an important basis for the ocean and marine biogeochemicall community downstream users (marine ecosystem modelers, climate services, etc.).
I only have 3 major comment and a set of minor comments/suggestions that aims to clarify some point of the paper.
Major comments:
Although the authors did a great job in assessing the performance of multiple models, several questions remain unaddressed while reading the manuscript.
- An in-depth process evaluation is needed
First, irrespective from the valuable effort (and outcome) of this work, that is evaluated marine biogeochemistry models embedded in CMIP6 ESMs, I think further attention to other physical drivers or control variables would be needed to better understand the performance of the models.
Another need would be to scrutinize inter-parameters relationship (chl-biomass, biomass-nitrate). I think the reader could be interested in further understanding of biases propagation across the marine biogeochemical cycles.
A similar attention would be needed also to dive a bit further into the model process parameterization. For instance, not all model simulates prognostically chlorophyll. This latter is derived from the carbon-to-chlorophyll ratio and phytoplankton biomass (see Séférian et al. 2020, who did an in-depth evaluation of model parameterization).
- The choice of biogeochemical tracers
The authors focused on key biogeochemical variables: chlorophyll, plankton biomass, nitrate and oxygen. All of these variables are analyzed at surface oceans — which makes sense for biological markers such as chlorophyll and plankton biomass but a surface analysis can hide model biases for oxygen and nitrate (oxycline and nutricline). There an expanded analysis including nitrate and oxygen profiles across the multi-model ensemble would be relevant.
I also think that phosphate and net marine productivity would be relevant to liase with global studies such as Kwiatkowski et al. 2020.
- Choice of reference datasets
Finally I’m also concerned by the use of the single CMEMS data are taken as ground truth observations although they aren’t.
As far as I am aware, CMEMS is a model-based data product. For the ocean physics the data product benefit from observation assimilation (it is thus a reanalysis) but for marine biogeochemistry it is only the marine biogeochemical model PISCES without any constraints.
I understand a focus on a given regional domain could pose challenge in terms of data availability. However, there are many high-resolution datasets for surface chlorophyll, net primary productivity etc. based on remote-sensing observation that can provide support to this work.
I would recommend to use them in addition to CMEMS. Indeed, past work (Lee et al. 2016) showed that this data product do not outcomes standard CMIP5 models when compared to true insitu observations.
Minor comments:
L9 why ? they are poorly constrained
L12 overestimations or underestimations, in what? (quantitative measures or score would be useful)
L23-25 This statement is inexact, recent work from Kwiatkowski et al 2020 shows that marine NPP in 2100 remains largely uncertain
L47 Consider to refer to Séférian et al. 2020 for the evaluation of global ESMs
L58 This aspect has to be analysed with caveats because for several bgc models chlorophyl is derived from phytoplankton biomass improving linkage with zooplankton response (see major comments)
L102 it would be nice to have a table of the references datasets used here
L112 CMEMS : it might be good to make it clear if CMEMS data is reanalysis and model reconstruction.
As far as I am aware of CMEMS relies on NEMO-PISCES+ assimilation for ocean hydrodynamics. PISCES, the marine bgc model adjust to improved physics but it is free (no assimilation). Therefore using CMEMS for evaluating marine bgc models is comparing models with another (more constrained) model.
L125 Table 1: I'm suprised to see PISCES (IPSL, CNRM, EC-Earth) and MARBL (CESM2) excluded from this exhaustive analysis — what are the reason ?
L129 Ranking: no cross-variable evaluation ? SST-Chlorophyll, etc. (see major comments)
L159-160 : please consider sharing scripts and data for this work (EGU journals recommandations)
L164-165 yes and no there are several paper that indicates how seasonal cycle can help to constrain projections (Behrenfeld et al. 2006, Kwiatkovski et al. 2017, etc..)
L183: typo: ensemble mean ?
‘Moderate’ needs to be quantified with skillscore
Figures 2-5: hard to see model differences.
I would present model baises against the reference
Figure 7: why for surface oxygen of MPI-based models outbest the others whereas it is not the case for the other variables ?
For these later MIROC-ES2L outbests the other models.
These differences needs to be explained.
My guess is that SST biaises in MPI-based models are much lower than the other models in this zone which explains why surface oxygen is better represented.
Behrenfeld, M. J., O'Malley, R. T., Siegel, D. A., McClain, C. R., Sarmiento, J. L., Feldman, G. C., Milligan, A. J., Falkowski, P. G., Letelier, R. M., and Boss, E. S.: Climate-driven trends in contemporary ocean productivity, Nature, 444, 752–755, https://doi.org/10.1038/nature05317, 2006.
Kwiatkowski, L., Torres, O., Bopp, L., Aumont, O., Chamberlain, M., Christian, J. R., Dunne, J. P., Gehlen, M., Ilyina, T., John, J. G., Lenton, A., Li, H., Lovenduski, N. S., Orr, J. C., Palmieri, J., Santana-Falcón, Y., Schwinger, J., Séférian, R., Stock, C. A., Tagliabue, A., Takano, Y., Tjiputra, J., Toyama, K., Tsujino, H., Watanabe, M., Yamamoto, A., Yool, A., and Ziehn, T.: Twenty-first century ocean warming, acidification, deoxygenation, and upper-ocean nutrient
Kwiatkowski, Bopp, Aumont, Ciais, Cox, Laufkötter, Li, Séférian: Emergent constraints on projections of declining primary production in the tropical oceans. Nature Climate Change 04/2017; 7(5)., DOI:10.1038/nclimate3265
and primary production decline from CMIP6 model projections, Biogeosciences, 17, 3439–3470, https://doi.org/10.5194/bg-17-3439-2020, 2020.
Séférian, R., Berthet, S., Yool, A., Palmiéri, J., Bopp, L., Tagliabue, A., Kwiatkowski, L., Aumont, O., Christian, J., Dunne, J., Gehlen, M., Ilyina, T., John, J. G., Li, H., Long, M., Luo, J. Y., Nakano, H., Romanou, A., Schwinger, J., Stock, C., Santana-Falcón, Y., Takano, Y., Tjiputra, J., Tsujino, H., Watanabe, M., Wu, T., Wu, F., Yamamoto, A. : Tracking improvement in simulated marine biogeochemistry between CMIP5 and CMIP6, Current Climate Change Reports, doi:10.1007/s40641-020-00160-0
Younjoo J. Lee, Patricia A. Matrai, Marjorie A. M. Friedrichs, Vincent S. Saba, Olivier Aumont, Marcel Babin, Erik T. Buitenhuis, Matthieu Chevallier, Lee de Mora, Morgane Dessert, John P. Dunne, Ingrid H. Ellingsen, Doron Feldman, Robert Frouin, Marion Gehlen, Thomas Gorgues, Tatiana Ilyina, Meibing Jin, Jasmin G. John, Jonathan Lawrence, Manfredi Manizza, Christophe Eugène Menkes, Coralie Perruche, Vincent Le Fouest, Ekaterina E. Popova, Anastasia Romanou, Annette Samuelsen, Jörg Schwinger, Séférian, R., Charles A. Stock, Jerry Tjiputra, L. Bruno Tremblay, Kyozo Ueyoshi, Marcello Vichi, Andrew Yool, Jinlun Zhang: Net primary productivity estimates and environmental variables in the Arctic Ocean: An assessment of coupled physical-biogeochemical models. Journal of Geophysical Research: Oceans 11/2016; 121(12)., DOI:10.1002/2016JC011993
Citation: https://doi.org/10.5194/egusphere-2024-72-RC1 -
AC1: 'Reply on RC1', Winfred Marshal, 22 May 2024
We would like to express our sincere gratitude for the time and effort you dedicated to reviewing our manuscript. Your insightful feedback and suggestions have improved our study and the clarity of the text. Please find attached the PDF file in which we have addressed all of the reviewer's comments.
-
RC2: 'Comment on egusphere-2024-72', Anonymous Referee #2, 08 May 2024
The comment was uploaded in the form of a supplement: https://egusphere.copernicus.org/preprints/2024/egusphere-2024-72/egusphere-2024-72-RC2-supplement.pdf
-
AC2: 'Reply on RC2', Winfred Marshal, 22 May 2024
We would like to express our sincere gratitude for the time and effort you dedicated to reviewing our manuscript. Your insightful feedback and suggestions have improved our study and the clarity of the text. Please find attached the PDF file in which we have addressed all of the reviewer's comments
-
AC2: 'Reply on RC2', Winfred Marshal, 22 May 2024
Interactive discussion
Status: closed
-
RC1: 'Comment on egusphere-2024-72', Anonymous Referee #1, 23 Apr 2024
In this manuscript, Marshal et al. scrutinize the performance of CMIP6 (state-of-the-art) Earth system models to simulate biogeochemical variables in the southern South China Sea (SCS).
.
The manuscript is clearly written and proposes a sound methodology. The results are well explained and discussed through the manuscript. This work presents an important basis for the ocean and marine biogeochemicall community downstream users (marine ecosystem modelers, climate services, etc.).
I only have 3 major comment and a set of minor comments/suggestions that aims to clarify some point of the paper.
Major comments:
Although the authors did a great job in assessing the performance of multiple models, several questions remain unaddressed while reading the manuscript.
- An in-depth process evaluation is needed
First, irrespective from the valuable effort (and outcome) of this work, that is evaluated marine biogeochemistry models embedded in CMIP6 ESMs, I think further attention to other physical drivers or control variables would be needed to better understand the performance of the models.
Another need would be to scrutinize inter-parameters relationship (chl-biomass, biomass-nitrate). I think the reader could be interested in further understanding of biases propagation across the marine biogeochemical cycles.
A similar attention would be needed also to dive a bit further into the model process parameterization. For instance, not all model simulates prognostically chlorophyll. This latter is derived from the carbon-to-chlorophyll ratio and phytoplankton biomass (see Séférian et al. 2020, who did an in-depth evaluation of model parameterization).
- The choice of biogeochemical tracers
The authors focused on key biogeochemical variables: chlorophyll, plankton biomass, nitrate and oxygen. All of these variables are analyzed at surface oceans — which makes sense for biological markers such as chlorophyll and plankton biomass but a surface analysis can hide model biases for oxygen and nitrate (oxycline and nutricline). There an expanded analysis including nitrate and oxygen profiles across the multi-model ensemble would be relevant.
I also think that phosphate and net marine productivity would be relevant to liase with global studies such as Kwiatkowski et al. 2020.
- Choice of reference datasets
Finally I’m also concerned by the use of the single CMEMS data are taken as ground truth observations although they aren’t.
As far as I am aware, CMEMS is a model-based data product. For the ocean physics the data product benefit from observation assimilation (it is thus a reanalysis) but for marine biogeochemistry it is only the marine biogeochemical model PISCES without any constraints.
I understand a focus on a given regional domain could pose challenge in terms of data availability. However, there are many high-resolution datasets for surface chlorophyll, net primary productivity etc. based on remote-sensing observation that can provide support to this work.
I would recommend to use them in addition to CMEMS. Indeed, past work (Lee et al. 2016) showed that this data product do not outcomes standard CMIP5 models when compared to true insitu observations.
Minor comments:
L9 why ? they are poorly constrained
L12 overestimations or underestimations, in what? (quantitative measures or score would be useful)
L23-25 This statement is inexact, recent work from Kwiatkowski et al 2020 shows that marine NPP in 2100 remains largely uncertain
L47 Consider to refer to Séférian et al. 2020 for the evaluation of global ESMs
L58 This aspect has to be analysed with caveats because for several bgc models chlorophyl is derived from phytoplankton biomass improving linkage with zooplankton response (see major comments)
L102 it would be nice to have a table of the references datasets used here
L112 CMEMS : it might be good to make it clear if CMEMS data is reanalysis and model reconstruction.
As far as I am aware of CMEMS relies on NEMO-PISCES+ assimilation for ocean hydrodynamics. PISCES, the marine bgc model adjust to improved physics but it is free (no assimilation). Therefore using CMEMS for evaluating marine bgc models is comparing models with another (more constrained) model.
L125 Table 1: I'm suprised to see PISCES (IPSL, CNRM, EC-Earth) and MARBL (CESM2) excluded from this exhaustive analysis — what are the reason ?
L129 Ranking: no cross-variable evaluation ? SST-Chlorophyll, etc. (see major comments)
L159-160 : please consider sharing scripts and data for this work (EGU journals recommandations)
L164-165 yes and no there are several paper that indicates how seasonal cycle can help to constrain projections (Behrenfeld et al. 2006, Kwiatkovski et al. 2017, etc..)
L183: typo: ensemble mean ?
‘Moderate’ needs to be quantified with skillscore
Figures 2-5: hard to see model differences.
I would present model baises against the reference
Figure 7: why for surface oxygen of MPI-based models outbest the others whereas it is not the case for the other variables ?
For these later MIROC-ES2L outbests the other models.
These differences needs to be explained.
My guess is that SST biaises in MPI-based models are much lower than the other models in this zone which explains why surface oxygen is better represented.
Behrenfeld, M. J., O'Malley, R. T., Siegel, D. A., McClain, C. R., Sarmiento, J. L., Feldman, G. C., Milligan, A. J., Falkowski, P. G., Letelier, R. M., and Boss, E. S.: Climate-driven trends in contemporary ocean productivity, Nature, 444, 752–755, https://doi.org/10.1038/nature05317, 2006.
Kwiatkowski, L., Torres, O., Bopp, L., Aumont, O., Chamberlain, M., Christian, J. R., Dunne, J. P., Gehlen, M., Ilyina, T., John, J. G., Lenton, A., Li, H., Lovenduski, N. S., Orr, J. C., Palmieri, J., Santana-Falcón, Y., Schwinger, J., Séférian, R., Stock, C. A., Tagliabue, A., Takano, Y., Tjiputra, J., Toyama, K., Tsujino, H., Watanabe, M., Yamamoto, A., Yool, A., and Ziehn, T.: Twenty-first century ocean warming, acidification, deoxygenation, and upper-ocean nutrient
Kwiatkowski, Bopp, Aumont, Ciais, Cox, Laufkötter, Li, Séférian: Emergent constraints on projections of declining primary production in the tropical oceans. Nature Climate Change 04/2017; 7(5)., DOI:10.1038/nclimate3265
and primary production decline from CMIP6 model projections, Biogeosciences, 17, 3439–3470, https://doi.org/10.5194/bg-17-3439-2020, 2020.
Séférian, R., Berthet, S., Yool, A., Palmiéri, J., Bopp, L., Tagliabue, A., Kwiatkowski, L., Aumont, O., Christian, J., Dunne, J., Gehlen, M., Ilyina, T., John, J. G., Li, H., Long, M., Luo, J. Y., Nakano, H., Romanou, A., Schwinger, J., Stock, C., Santana-Falcón, Y., Takano, Y., Tjiputra, J., Tsujino, H., Watanabe, M., Wu, T., Wu, F., Yamamoto, A. : Tracking improvement in simulated marine biogeochemistry between CMIP5 and CMIP6, Current Climate Change Reports, doi:10.1007/s40641-020-00160-0
Younjoo J. Lee, Patricia A. Matrai, Marjorie A. M. Friedrichs, Vincent S. Saba, Olivier Aumont, Marcel Babin, Erik T. Buitenhuis, Matthieu Chevallier, Lee de Mora, Morgane Dessert, John P. Dunne, Ingrid H. Ellingsen, Doron Feldman, Robert Frouin, Marion Gehlen, Thomas Gorgues, Tatiana Ilyina, Meibing Jin, Jasmin G. John, Jonathan Lawrence, Manfredi Manizza, Christophe Eugène Menkes, Coralie Perruche, Vincent Le Fouest, Ekaterina E. Popova, Anastasia Romanou, Annette Samuelsen, Jörg Schwinger, Séférian, R., Charles A. Stock, Jerry Tjiputra, L. Bruno Tremblay, Kyozo Ueyoshi, Marcello Vichi, Andrew Yool, Jinlun Zhang: Net primary productivity estimates and environmental variables in the Arctic Ocean: An assessment of coupled physical-biogeochemical models. Journal of Geophysical Research: Oceans 11/2016; 121(12)., DOI:10.1002/2016JC011993
Citation: https://doi.org/10.5194/egusphere-2024-72-RC1 -
AC1: 'Reply on RC1', Winfred Marshal, 22 May 2024
We would like to express our sincere gratitude for the time and effort you dedicated to reviewing our manuscript. Your insightful feedback and suggestions have improved our study and the clarity of the text. Please find attached the PDF file in which we have addressed all of the reviewer's comments.
-
RC2: 'Comment on egusphere-2024-72', Anonymous Referee #2, 08 May 2024
The comment was uploaded in the form of a supplement: https://egusphere.copernicus.org/preprints/2024/egusphere-2024-72/egusphere-2024-72-RC2-supplement.pdf
-
AC2: 'Reply on RC2', Winfred Marshal, 22 May 2024
We would like to express our sincere gratitude for the time and effort you dedicated to reviewing our manuscript. Your insightful feedback and suggestions have improved our study and the clarity of the text. Please find attached the PDF file in which we have addressed all of the reviewer's comments
-
AC2: 'Reply on RC2', Winfred Marshal, 22 May 2024
Peer review completion
Journal article(s) based on this preprint
Viewed
| HTML | XML | Total | BibTeX | EndNote | |
|---|---|---|---|---|---|
| 345 | 88 | 24 | 457 | 19 | 12 |
- HTML: 345
- PDF: 88
- XML: 24
- Total: 457
- BibTeX: 19
- EndNote: 12
Viewed (geographical distribution)
| Country | # | Views | % |
|---|---|---|---|
| United States of America | 1 | 115 | 24 |
| Malaysia | 2 | 95 | 19 |
| China | 3 | 53 | 11 |
| Germany | 4 | 28 | 5 |
| United Kingdom | 5 | 25 | 5 |
| Total: | 0 |
| HTML: | 0 |
| PDF: | 0 |
| XML: | 0 |
- 1
- 115
Winfred Marshal
Mohd Fadzil Bin Mohd Akhir
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
- Preprint
(1989 KB) - Metadata XML