15 Aug 2022
15 Aug 2022
Status: this preprint is open for discussion.

Quality Assessment of Meta-Analyses on Soil Organic Carbon

Julia Fohrafellner1, Sophie Zechmeister-Boltenstern2, Rajasekaran Murugan2, and Elena Valkama3 Julia Fohrafellner et al.
  • 1BIOS Science Austria, Dietrichgasse 27, 1130 Vienna, Austria
  • 2Institute of Soil Research (IBF), Department of Forest- and Soil Sciences, University of Natural Resources and Life Sciences, Vienna (BOKU), Gregor-Mendel-Straße 33, 1180 Vienna, Austria
  • 3Natural Resources Institute Finland (Luke), Bioeconomy and environment, Sustainability Science and Indicators. Tietotie 4, 31600 Jokioinen, Finland

Abstract. Soil organic carbon (SOC) plays a vital role in the global carbon cycle and is a potential sink for carbon dioxide. Agricultural management practices can support carbon sequestration and therefore offer potential removal strategies, whilst improving overall soil quality. Meta-analysis allows to summarize results from primary articles by calculating an overall effect size and hence to reveal the source of variation across studies. The number of meta-analyses published in the field of agriculture is continuously rising. At the same time, more and more articles refer to their synthesis work as a meta-analysis, despite applying less than rigorous methodologies. As a result, poor quality meta-analyses are published, which may lead to questionable conclusions and recommendations to scientists, policymakers and farmers.

This study aims at quantitatively analyzing 31 meta-analyses, published between the years 2005–2020, studying the effects of different management practices on SOC. We compiled a quality criteria-set, suitable for soil and agricultural sciences, by adapting existing meta-analytical guidelines from other disciplines. The set is supported by a scoring scheme, which allows a quantitative analysis. The retrieved meta-analyses were structured according to 11 management categories, such as tillage, cover crops, residue management, biochar application etc., which allowed us to assess the state-of-knowledge on these categories. Major deficiencies were found in the use of standard metrics for effect size calculation, independence of effect sizes, standard deviation extraction for each study and weighting by the inverse of variance. Only one out of 31 SOC meta-analyses, which studied the effects of no-till/reduced tillage compared to conventional tillage, was found to be of high quality. Therefore, improved meta-analyses on the effects of e.g., organic agriculture, biochar, fertilization or crop diversification on SOC are urgently needed.

We conclude that, despite the efforts over the last 15 years, the quality of meta-analyses on SOC research is still low. In order for the scientific community to provide high quality synthesis work and to make advancements in the sustainable management of agricultural soils, we need to adapt rigorous methodologies of meta-analysis as quickly as possible.

Julia Fohrafellner et al.

Status: open (until 01 Oct 2022)

Comment types: AC – author | RC – referee | CC – community | EC – editor | CEC – chief editor | : Report abuse
  • CC1: 'Comment on egusphere-2022-488', Tommaso Tadiello, 25 Aug 2022 reply
  • CC2: 'Comment on egusphere-2022-488', Damien Beillouin, 29 Aug 2022 reply
  • RC1: 'Comment on egusphere-2022-488', John Koestel, 09 Sep 2022 reply
  • CC3: 'Comment on egusphere-2022-488', Marco Acutis, 30 Sep 2022 reply

Julia Fohrafellner et al.

Julia Fohrafellner et al.


Total article views: 463 (including HTML, PDF, and XML)
HTML PDF XML Total BibTeX EndNote
326 117 20 463 4 3
  • HTML: 326
  • PDF: 117
  • XML: 20
  • Total: 463
  • BibTeX: 4
  • EndNote: 3
Views and downloads (calculated since 15 Aug 2022)
Cumulative views and downloads (calculated since 15 Aug 2022)

Viewed (geographical distribution)

Total article views: 441 (including HTML, PDF, and XML) Thereof 441 with geography defined and 0 with unknown origin.
Country # Views %
  • 1
Latest update: 30 Sep 2022
Short summary
The number of meta-analyses in agriculture and soil sciences is continuously rising, often being of poor quality. We quantitatively analyzed the quality of 31 meta-analyses, studying the effects of different management practices on soil organic carbon (SOC). We conclude that only one meta-analysis on no-till/reduced tillage reached a high score. Either new or improved meta-analyses are urgently needed on the effects of organic agriculture, biochar, fertilization and crop diversification on SOC.