Evaluation of CMIP6 Models Performance in Simulating Historical Biogeochemistry across Southern South China Sea

Marshal, Winfred; Chung, Jing Xiang; Mohd Akhir, Mohd Fadzil Bin

doi:https://doi.org/10.5194/egusphere-2024-72

Preprints

https://doi.org/10.5194/egusphere-2024-72

Preprints

28 Feb 2024

| 28 Feb 2024

Evaluation of CMIP6 Models Performance in Simulating Historical Biogeochemistry across Southern South China Sea

Winfred Marshal, Jing Xiang Chung, and Mohd Fadzil Bin Mohd Akhir

Abstract. This study evaluates the ability of Earth System Models (ESMs) from the Coupled Model Intercomparison Project Phase 6 (CMIP6) to simulate biogeochemical variables in the southern South China Sea (SCS). The analysis focuses on chlorophyll, phytoplankton, nitrate, and oxygen at annual and seasonal scales. The models' performance is assessed against Copernicus Marine Environment Monitoring Service (CMEMS) data using statistical metrics such as the Taylor diagram and Taylor skill score. The results show that the models generally capture the observed spatial patterns of biogeochemical variables but exhibit varying degrees of overestimation or underestimation. The performance of the models is also influenced by the season, with some models showing better performance during the southwest monsoon than the northeast monsoon. Overall, the top five best-performing models for biogeochemical variables are MIROC-ES2H, GFDL-ESM4, CanESM5-CanOE, MPI-ESM1-2-LR, and NorESM2-LM. The findings of this study have implications for researchers and end-users of the datasets, providing guidance for model improvement and understanding the impacts of climate change on the SCS ecosystem.

Received: 10 Jan 2024 – Discussion started: 28 Feb 2024

Publisher's note: Copernicus Publications remains neutral with regard to jurisdictional claims made in the text, published maps, institutional affiliations, or any other geographical representation in this preprint. The responsibility to include appropriate place names lies with the authors.

Download & links

Preprint (PDF, 1989 KB)

Notice on discussion status
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
Preprint (1989 KB)

Download & links

The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.

Journal article(s) based on this preprint

13 Sep 2024

Evaluation of CMIP6 model performance in simulating historical biogeochemistry across the southern South China Sea

Winfred Marshal, Jing Xiang Chung, Nur Hidayah Roseli, Roswati Md Amin, and Mohd Fadzil Bin Mohd Akhir

Biogeosciences, 21, 4007–4035, https://doi.org/10.5194/bg-21-4007-2024,https://doi.org/10.5194/bg-21-4007-2024, 2024

Short summary

Winfred Marshal, Jing Xiang Chung, and Mohd Fadzil Bin Mohd Akhir

Interactive discussion

Status: closed

RC1:
'Comment on egusphere-2024-72', Anonymous Referee #1, 23 Apr 2024
In this manuscript, Marshal et al. scrutinize the performance of CMIP6 (state-of-the-art) Earth system models to simulate biogeochemical variables in the southern South China Sea (SCS).
.
The manuscript is clearly written and proposes a sound methodology. The results are well explained and discussed through the manuscript. This work presents an important basis for the ocean and marine biogeochemicall community downstream users (marine ecosystem modelers, climate services, etc.).
I only have 3 major comment and a set of minor comments/suggestions that aims to clarify some point of the paper.
Major comments:

Although the authors did a great job in assessing the performance of multiple models, several questions remain unaddressed while reading the manuscript.

An in-depth process evaluation is needed

First, irrespective from the valuable effort (and outcome) of this work, that is evaluated marine biogeochemistry models embedded in CMIP6 ESMs, I think further attention to other physical drivers or control variables would be needed to better understand the performance of the models.
Another need would be to scrutinize inter-parameters relationship (chl-biomass, biomass-nitrate). I think the reader could be interested in further understanding of biases propagation across the marine biogeochemical cycles.
A similar attention would be needed also to dive a bit further into the model process parameterization. For instance, not all model simulates prognostically chlorophyll. This latter is derived from the carbon-to-chlorophyll ratio and phytoplankton biomass (see Séférian et al. 2020, who did an in-depth evaluation of model parameterization).

The choice of biogeochemical tracers

The authors focused on key biogeochemical variables: chlorophyll, plankton biomass, nitrate and oxygen. All of these variables are analyzed at surface oceans — which makes sense for biological markers such as chlorophyll and plankton biomass but a surface analysis can hide model biases for oxygen and nitrate (oxycline and nutricline). There an expanded analysis including nitrate and oxygen profiles across the multi-model ensemble would be relevant.
I also think that phosphate and net marine productivity would be relevant to liase with global studies such as Kwiatkowski et al. 2020.

Choice of reference datasets

Finally I’m also concerned by the use of the single CMEMS data are taken as ground truth observations although they aren’t.
As far as I am aware, CMEMS is a model-based data product. For the ocean physics the data product benefit from observation assimilation (it is thus a reanalysis) but for marine biogeochemistry it is only the marine biogeochemical model PISCES without any constraints.
I understand a focus on a given regional domain could pose challenge in terms of data availability. However, there are many high-resolution datasets for surface chlorophyll, net primary productivity etc. based on remote-sensing observation that can provide support to this work.
I would recommend to use them in addition to CMEMS. Indeed, past work (Lee et al. 2016) showed that this data product do not outcomes standard CMIP5 models when compared to true insitu observations.

Minor comments:
L9 why ? they are poorly constrained
L12 overestimations or underestimations, in what? (quantitative measures or score would be useful)
L23-25 This statement is inexact, recent work from Kwiatkowski et al 2020 shows that marine NPP in 2100 remains largely uncertain
L47 Consider to refer to Séférian et al. 2020 for the evaluation of global ESMs
L58 This aspect has to be analysed with caveats because for several bgc models chlorophyl is derived from phytoplankton biomass improving linkage with zooplankton response (see major comments)
L102 it would be nice to have a table of the references datasets used here
L112 CMEMS : it might be good to make it clear if CMEMS data is reanalysis and model reconstruction.
As far as I am aware of CMEMS relies on NEMO-PISCES+ assimilation for ocean hydrodynamics. PISCES, the marine bgc model adjust to improved physics but it is free (no assimilation). Therefore using CMEMS for evaluating marine bgc models is comparing models with another (more constrained) model.
L125 Table 1: I'm suprised to see PISCES (IPSL, CNRM, EC-Earth) and MARBL (CESM2) excluded from this exhaustive analysis — what are the reason ?
L129 Ranking: no cross-variable evaluation ? SST-Chlorophyll, etc. (see major comments)
L159-160 : please consider sharing scripts and data for this work (EGU journals recommandations)
L164-165 yes and no there are several paper that indicates how seasonal cycle can help to constrain projections (Behrenfeld et al. 2006, Kwiatkovski et al. 2017, etc..)
L183: typo: ensemble mean ?
‘Moderate’ needs to be quantified with skillscore

Figures 2-5: hard to see model differences.
I would present model baises against the reference

Figure 7: why for surface oxygen of MPI-based models outbest the others whereas it is not the case for the other variables ?
For these later MIROC-ES2L outbests the other models.
These differences needs to be explained.
My guess is that SST biaises in MPI-based models are much lower than the other models in this zone which explains why surface oxygen is better represented.

Behrenfeld, M. J., O'Malley, R. T., Siegel, D. A., McClain, C. R., Sarmiento, J. L., Feldman, G. C., Milligan, A. J., Falkowski, P. G., Letelier, R. M., and Boss, E. S.: Climate-driven trends in contemporary ocean productivity, Nature, 444, 752–755, https://doi.org/10.1038/nature05317, 2006.

Kwiatkowski, L., Torres, O., Bopp, L., Aumont, O., Chamberlain, M., Christian, J. R., Dunne, J. P., Gehlen, M., Ilyina, T., John, J. G., Lenton, A., Li, H., Lovenduski, N. S., Orr, J. C., Palmieri, J., Santana-Falcón, Y., Schwinger, J., Séférian, R., Stock, C. A., Tagliabue, A., Takano, Y., Tjiputra, J., Toyama, K., Tsujino, H., Watanabe, M., Yamamoto, A., Yool, A., and Ziehn, T.: Twenty-first century ocean warming, acidification, deoxygenation, and upper-ocean nutrient
Kwiatkowski, Bopp, Aumont, Ciais, Cox, Laufkötter, Li, Séférian: Emergent constraints on projections of declining primary production in the tropical oceans. Nature Climate Change 04/2017; 7(5)., DOI:10.1038/nclimate3265
and primary production decline from CMIP6 model projections, Biogeosciences, 17, 3439–3470, https://doi.org/10.5194/bg-17-3439-2020, 2020.

Séférian, R., Berthet, S., Yool, A., Palmiéri, J., Bopp, L., Tagliabue, A., Kwiatkowski, L., Aumont, O., Christian, J., Dunne, J., Gehlen, M., Ilyina, T., John, J. G., Li, H., Long, M., Luo, J. Y., Nakano, H., Romanou, A., Schwinger, J., Stock, C., Santana-Falcón, Y., Takano, Y., Tjiputra, J., Tsujino, H., Watanabe, M., Wu, T., Wu, F., Yamamoto, A. : Tracking improvement in simulated marine biogeochemistry between CMIP5 and CMIP6, Current Climate Change Reports, doi:10.1007/s40641-020-00160-0

Younjoo J. Lee, Patricia A. Matrai, Marjorie A. M. Friedrichs, Vincent S. Saba, Olivier Aumont, Marcel Babin, Erik T. Buitenhuis, Matthieu Chevallier, Lee de Mora, Morgane Dessert, John P. Dunne, Ingrid H. Ellingsen, Doron Feldman, Robert Frouin, Marion Gehlen, Thomas Gorgues, Tatiana Ilyina, Meibing Jin, Jasmin G. John, Jonathan Lawrence, Manfredi Manizza, Christophe Eugène Menkes, Coralie Perruche, Vincent Le Fouest, Ekaterina E. Popova, Anastasia Romanou, Annette Samuelsen, Jörg Schwinger, Séférian, R., Charles A. Stock, Jerry Tjiputra, L. Bruno Tremblay, Kyozo Ueyoshi, Marcello Vichi, Andrew Yool, Jinlun Zhang: Net primary productivity estimates and environmental variables in the Arctic Ocean: An assessment of coupled physical-biogeochemical models. Journal of Geophysical Research: Oceans 11/2016; 121(12)., DOI:10.1002/2016JC011993
Citation: https://doi.org/10.5194/egusphere-2024-72-RC1
- AC1: 'Reply on RC1', Winfred Marshal, 22 May 2024
  
  We would like to express our sincere gratitude for the time and effort you dedicated to reviewing our manuscript. Your insightful feedback and suggestions have improved our study and the clarity of the text. Please find attached the PDF file in which we have addressed all of the reviewer's comments.
  
  Citation: https://doi.org/10.5194/egusphere-2024-72-AC1
RC2:
'Comment on egusphere-2024-72', Anonymous Referee #2, 08 May 2024

The comment was uploaded in the form of a supplement: https://egusphere.copernicus.org/preprints/2024/egusphere-2024-72/egusphere-2024-72-RC2-supplement.pdf

Citation: https://doi.org/10.5194/egusphere-2024-72-RC2
- AC2: 'Reply on RC2', Winfred Marshal, 22 May 2024
  
  We would like to express our sincere gratitude for the time and effort you dedicated to reviewing our manuscript. Your insightful feedback and suggestions have improved our study and the clarity of the text. Please find attached the PDF file in which we have addressed all of the reviewer's comments
  
  Citation: https://doi.org/10.5194/egusphere-2024-72-AC2

Interactive discussion

Status: closed

RC1:
'Comment on egusphere-2024-72', Anonymous Referee #1, 23 Apr 2024
In this manuscript, Marshal et al. scrutinize the performance of CMIP6 (state-of-the-art) Earth system models to simulate biogeochemical variables in the southern South China Sea (SCS).
.
The manuscript is clearly written and proposes a sound methodology. The results are well explained and discussed through the manuscript. This work presents an important basis for the ocean and marine biogeochemicall community downstream users (marine ecosystem modelers, climate services, etc.).
I only have 3 major comment and a set of minor comments/suggestions that aims to clarify some point of the paper.
Major comments:

Although the authors did a great job in assessing the performance of multiple models, several questions remain unaddressed while reading the manuscript.

An in-depth process evaluation is needed

First, irrespective from the valuable effort (and outcome) of this work, that is evaluated marine biogeochemistry models embedded in CMIP6 ESMs, I think further attention to other physical drivers or control variables would be needed to better understand the performance of the models.
Another need would be to scrutinize inter-parameters relationship (chl-biomass, biomass-nitrate). I think the reader could be interested in further understanding of biases propagation across the marine biogeochemical cycles.
A similar attention would be needed also to dive a bit further into the model process parameterization. For instance, not all model simulates prognostically chlorophyll. This latter is derived from the carbon-to-chlorophyll ratio and phytoplankton biomass (see Séférian et al. 2020, who did an in-depth evaluation of model parameterization).

The choice of biogeochemical tracers

The authors focused on key biogeochemical variables: chlorophyll, plankton biomass, nitrate and oxygen. All of these variables are analyzed at surface oceans — which makes sense for biological markers such as chlorophyll and plankton biomass but a surface analysis can hide model biases for oxygen and nitrate (oxycline and nutricline). There an expanded analysis including nitrate and oxygen profiles across the multi-model ensemble would be relevant.
I also think that phosphate and net marine productivity would be relevant to liase with global studies such as Kwiatkowski et al. 2020.

Choice of reference datasets

Finally I’m also concerned by the use of the single CMEMS data are taken as ground truth observations although they aren’t.
As far as I am aware, CMEMS is a model-based data product. For the ocean physics the data product benefit from observation assimilation (it is thus a reanalysis) but for marine biogeochemistry it is only the marine biogeochemical model PISCES without any constraints.
I understand a focus on a given regional domain could pose challenge in terms of data availability. However, there are many high-resolution datasets for surface chlorophyll, net primary productivity etc. based on remote-sensing observation that can provide support to this work.
I would recommend to use them in addition to CMEMS. Indeed, past work (Lee et al. 2016) showed that this data product do not outcomes standard CMIP5 models when compared to true insitu observations.

Minor comments:
L9 why ? they are poorly constrained
L12 overestimations or underestimations, in what? (quantitative measures or score would be useful)
L23-25 This statement is inexact, recent work from Kwiatkowski et al 2020 shows that marine NPP in 2100 remains largely uncertain
L47 Consider to refer to Séférian et al. 2020 for the evaluation of global ESMs
L58 This aspect has to be analysed with caveats because for several bgc models chlorophyl is derived from phytoplankton biomass improving linkage with zooplankton response (see major comments)
L102 it would be nice to have a table of the references datasets used here
L112 CMEMS : it might be good to make it clear if CMEMS data is reanalysis and model reconstruction.
As far as I am aware of CMEMS relies on NEMO-PISCES+ assimilation for ocean hydrodynamics. PISCES, the marine bgc model adjust to improved physics but it is free (no assimilation). Therefore using CMEMS for evaluating marine bgc models is comparing models with another (more constrained) model.
L125 Table 1: I'm suprised to see PISCES (IPSL, CNRM, EC-Earth) and MARBL (CESM2) excluded from this exhaustive analysis — what are the reason ?
L129 Ranking: no cross-variable evaluation ? SST-Chlorophyll, etc. (see major comments)
L159-160 : please consider sharing scripts and data for this work (EGU journals recommandations)
L164-165 yes and no there are several paper that indicates how seasonal cycle can help to constrain projections (Behrenfeld et al. 2006, Kwiatkovski et al. 2017, etc..)
L183: typo: ensemble mean ?
‘Moderate’ needs to be quantified with skillscore

Figures 2-5: hard to see model differences.
I would present model baises against the reference

Figure 7: why for surface oxygen of MPI-based models outbest the others whereas it is not the case for the other variables ?
For these later MIROC-ES2L outbests the other models.
These differences needs to be explained.
My guess is that SST biaises in MPI-based models are much lower than the other models in this zone which explains why surface oxygen is better represented.

Behrenfeld, M. J., O'Malley, R. T., Siegel, D. A., McClain, C. R., Sarmiento, J. L., Feldman, G. C., Milligan, A. J., Falkowski, P. G., Letelier, R. M., and Boss, E. S.: Climate-driven trends in contemporary ocean productivity, Nature, 444, 752–755, https://doi.org/10.1038/nature05317, 2006.

Kwiatkowski, L., Torres, O., Bopp, L., Aumont, O., Chamberlain, M., Christian, J. R., Dunne, J. P., Gehlen, M., Ilyina, T., John, J. G., Lenton, A., Li, H., Lovenduski, N. S., Orr, J. C., Palmieri, J., Santana-Falcón, Y., Schwinger, J., Séférian, R., Stock, C. A., Tagliabue, A., Takano, Y., Tjiputra, J., Toyama, K., Tsujino, H., Watanabe, M., Yamamoto, A., Yool, A., and Ziehn, T.: Twenty-first century ocean warming, acidification, deoxygenation, and upper-ocean nutrient
Kwiatkowski, Bopp, Aumont, Ciais, Cox, Laufkötter, Li, Séférian: Emergent constraints on projections of declining primary production in the tropical oceans. Nature Climate Change 04/2017; 7(5)., DOI:10.1038/nclimate3265
and primary production decline from CMIP6 model projections, Biogeosciences, 17, 3439–3470, https://doi.org/10.5194/bg-17-3439-2020, 2020.

Séférian, R., Berthet, S., Yool, A., Palmiéri, J., Bopp, L., Tagliabue, A., Kwiatkowski, L., Aumont, O., Christian, J., Dunne, J., Gehlen, M., Ilyina, T., John, J. G., Li, H., Long, M., Luo, J. Y., Nakano, H., Romanou, A., Schwinger, J., Stock, C., Santana-Falcón, Y., Takano, Y., Tjiputra, J., Tsujino, H., Watanabe, M., Wu, T., Wu, F., Yamamoto, A. : Tracking improvement in simulated marine biogeochemistry between CMIP5 and CMIP6, Current Climate Change Reports, doi:10.1007/s40641-020-00160-0

Younjoo J. Lee, Patricia A. Matrai, Marjorie A. M. Friedrichs, Vincent S. Saba, Olivier Aumont, Marcel Babin, Erik T. Buitenhuis, Matthieu Chevallier, Lee de Mora, Morgane Dessert, John P. Dunne, Ingrid H. Ellingsen, Doron Feldman, Robert Frouin, Marion Gehlen, Thomas Gorgues, Tatiana Ilyina, Meibing Jin, Jasmin G. John, Jonathan Lawrence, Manfredi Manizza, Christophe Eugène Menkes, Coralie Perruche, Vincent Le Fouest, Ekaterina E. Popova, Anastasia Romanou, Annette Samuelsen, Jörg Schwinger, Séférian, R., Charles A. Stock, Jerry Tjiputra, L. Bruno Tremblay, Kyozo Ueyoshi, Marcello Vichi, Andrew Yool, Jinlun Zhang: Net primary productivity estimates and environmental variables in the Arctic Ocean: An assessment of coupled physical-biogeochemical models. Journal of Geophysical Research: Oceans 11/2016; 121(12)., DOI:10.1002/2016JC011993
Citation: https://doi.org/10.5194/egusphere-2024-72-RC1
- AC1: 'Reply on RC1', Winfred Marshal, 22 May 2024
  
  We would like to express our sincere gratitude for the time and effort you dedicated to reviewing our manuscript. Your insightful feedback and suggestions have improved our study and the clarity of the text. Please find attached the PDF file in which we have addressed all of the reviewer's comments.
  
  Citation: https://doi.org/10.5194/egusphere-2024-72-AC1
RC2:
'Comment on egusphere-2024-72', Anonymous Referee #2, 08 May 2024

The comment was uploaded in the form of a supplement: https://egusphere.copernicus.org/preprints/2024/egusphere-2024-72/egusphere-2024-72-RC2-supplement.pdf

Citation: https://doi.org/10.5194/egusphere-2024-72-RC2
- AC2: 'Reply on RC2', Winfred Marshal, 22 May 2024
  
  We would like to express our sincere gratitude for the time and effort you dedicated to reviewing our manuscript. Your insightful feedback and suggestions have improved our study and the clarity of the text. Please find attached the PDF file in which we have addressed all of the reviewer's comments
  
  Citation: https://doi.org/10.5194/egusphere-2024-72-AC2

Peer review completion

AR: Author's response | RR: Referee report | ED: Editor decision | EF: Editorial file upload

ED: Reconsider after major revisions (29 May 2024) by Carolin Löscher

AR by Winfred Marshal on behalf of the Authors (31 May 2024) Author's response Author's tracked changes Manuscript

ED: Referee Nomination & Report Request started (04 Jun 2024) by Carolin Löscher

RR by Anonymous Referee #1 (18 Jun 2024)

RR by Anonymous Referee #2 (22 Jun 2024)

ED: Publish subject to minor revisions (review by editor) (03 Jul 2024) by Carolin Löscher

AR by Winfred Marshal on behalf of the Authors (12 Jul 2024) Author's response Author's tracked changes Manuscript

ED: Publish as is (31 Jul 2024) by Carolin Löscher

AR by Winfred Marshal on behalf of the Authors (02 Aug 2024)

Journal article(s) based on this preprint

13 Sep 2024

Evaluation of CMIP6 model performance in simulating historical biogeochemistry across the southern South China Sea

Winfred Marshal, Jing Xiang Chung, Nur Hidayah Roseli, Roswati Md Amin, and Mohd Fadzil Bin Mohd Akhir

Biogeosciences, 21, 4007–4035, https://doi.org/10.5194/bg-21-4007-2024,https://doi.org/10.5194/bg-21-4007-2024, 2024

Short summary

Winfred Marshal, Jing Xiang Chung, and Mohd Fadzil Bin Mohd Akhir

Viewed

Total article views: 457 (including HTML, PDF, and XML)

HTML	PDF	XML	Total	BibTeX	EndNote
345	88	24	457	19	12

HTML: 345
PDF: 88
XML: 24
Total: 457
BibTeX: 19
EndNote: 12

Views and downloads (calculated since 28 Feb 2024)

Month	HTML	PDF	XML	Total
Feb 2024	46	10	3	59
Mar 2024	71	11	1	83
Apr 2024	46	12	6	64
May 2024	76	21	6	103
Jun 2024	31	12	3	46
Jul 2024	32	17	4	53
Aug 2024	28	3	1	32
Sep 2024	15	2	0	17

Cumulative views and downloads (calculated since 28 Feb 2024)

Month	HTML	PDF	XML	Total
Feb 2024	46	10	3	59
Mar 2024	71	11	1	83
Apr 2024	46	12	6	64
May 2024	76	21	6	103
Jun 2024	31	12	3	46
Jul 2024	32	17	4	53
Aug 2024	28	3	1	32
Sep 2024	15	2	0	17

Viewed (geographical distribution)

Total article views: 478 (including HTML, PDF, and XML) Thereof 478 with geography defined and 0 with unknown origin.

Country	#	Views	%

Latest update: 13 Sep 2024

Download

The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.

Preprint (1989 KB)
Metadata XML

Short summary

This study stands out for thoroughly examining CMIP6 ESMs ability to simulate biogeochemical variables in the southern South China Sea, an economically important region. It assesses variables like chlorophyll, phytoplankton, nitrate and oxygen on annual and seasonal scales. While global assessments exist, this study addresses a gap by objectively ranking 13 CMIP6 ocean biogeochemistry models' performance at a regional level, focusing on replicating specific observed biogeochemical variables.


Total:	0
HTML:	0
PDF:	0
XML:	0

Evaluation of CMIP6 Models Performance in Simulating Historical Biogeochemistry across Southern South China Sea

Journal article(s) based on this preprint

Interactive discussion

Interactive discussion

Peer review completion

Suggestions for revision or reasons for rejection

Journal article(s) based on this preprint

Viewed

Viewed (geographical distribution)