the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
| 10 Jun 2025
MeteoSaver v1.0: a machine-learning based software for the transcription of historical weather data
Abstract. Archives of observed weather data present unique opportunities for scientists to obtain long time series of the historical climate for many regions of the world. Unfortunately, most of these observational records are to-date available only on paper, and thus require digitization and transcription to facilitate analysis of climatic trends. Here we present a new open-source software, MeteoSaver, that uses machine learning (ML) algorithms to transcribe handwritten records of historical weather data. MeteoSaver version 1.0 processes images of tabular sheets alongside user-defined configuration settings, performing transcription through five sequential steps: (i) image pre-processing, (ii) table and cell detection, (iii) transcription, (iv) quality assessment and quality control, and (v) data formatting and upload. As an illustration and evaluation of the software, we apply MeteoSaver to ten pictured sheets of handwritten temperature observations from the Democratic Republic of the Congo. The results show that 95–100 % of the records can be transcribed, of which a median of 74.4 % reached the highest internal quality flag and 74 % matches with the manually transcribed record, yielding a median mean absolute error of 0.3 °C. These results illustrate that MeteoSaver can be applied to a range of handwriting styles and varying tabular dimensions, paper sizes, and maintenance conditions, highlighting its potential for transcribing tabular meteorological observations from multiple regions, especially if the sheets have a consistent format. Overall, our open-source software can help address the challenges of limited available hydroclimatic data within many regions of the world, by helping to save millions of handwritten records of historical weather data presently stored in archives, and expedite research on the climate and environmental changes in data scarce regions.
Competing interests: At least one of the (co-)authors is a member of the editorial board of Geoscientific Model Development.
Publisher's note: Copernicus Publications remains neutral with regard to jurisdictional claims made in the text, published maps, institutional affiliations, or any other geographical representation in this paper. While Copernicus Publications makes every effort to include appropriate place names, the final responsibility lies with the authors. Views expressed in the text are those of the authors and do not necessarily reflect the views of the publisher.- Preprint
(38330 KB) - Metadata XML
-
Supplement
(387 KB) - BibTeX
- EndNote
Status: final response (author comments only)
- RC1: 'Comment on egusphere-2024-3779', Anonymous Referee #1, 15 Feb 2026
-
RC2: 'Comment on egusphere-2024-3779', Chris Lennard, 20 Feb 2026
I commend this work that has developed an automated procedure to transcribe and digitise tabular meteorological observations, particularly in data sparse regions where weather data are currently stored in archives. If manually done, this is an enormously onerous and thankless task so an automated procedure, such as the one presented here, is an invaluable tool in the data rescue endeavour.
Technically, each progressive step of the MeteoSaver v1.0 package is logical, building on the previous step and in each step the quality control checks are thorough and account for errors in the transcription as well as common errors found in recorded station data.
The quality control checks of the resultant transcribed data are those generally used to assess the quality of temperature data. This seems to be by design and is understandable as temperature is a relatively homogeneous variable to test the automatic transcription methodology and quality control procedures on.
However, in the title of the paper and elsewhere in the manuscript the phrase "Weather data” is used, which implicitly includes rainfall (and other data). In lines 444 - 449 the authors mention variables other than temperature but I note rainfall is missing in this list. I would suggest rainfall is an extremely important variable to transcribe, particularly in currently data-sparse regions, given the large observational uncertainties in these regions as described in the Introduction. The observation data sheets presented in the paper include rainfall so I would like to understand why it does not seem to be a variable being considered for transcription.
I would therefore be grateful for the authors to include a paragraph or two about why rainfall is not considered in the paper, and if lines 444 - 449 are an indication of the variables to be considered in later versions of the software, why rainfall does not feature. I’m not asking the authors to present rainfall in the paper as has been done for temperature, only an explanation of why (it may be more complex a task).
While I do realise this is Version 1 of the software, I also recognise the enormous potential it presents to reduce the huge uncertainties in rainfall variability and trend in data sparse regions, particularly in Africa.
Lastly, given the focus on temperature, perhaps constrain the title of the paper to temperature.
My congratulations and thanks again to the authors for developing this valuable tool that will make it easier to transcribe paper-based weather data for use in digital observational datasets. It has the potential to reduce observational uncertainties in currently data sparse areas, including the DRC.
Citation: https://doi.org/10.5194/egusphere-2024-3779-RC2
Model code and software
MeteoSaver v1.0 Derrick Muheki, Bas Vercruysse, Krishna Kumar Thirukokaranam Chandrasekar, Koen Hufkens, and Wim Thiery https://doi.org/10.5281/zenodo.14246037
Viewed
| HTML | XML | Total | Supplement | BibTeX | EndNote | |
|---|---|---|---|---|---|---|
| 2,987 | 411 | 55 | 3,453 | 43 | 40 | 54 |
- HTML: 2,987
- PDF: 411
- XML: 55
- Total: 3,453
- Supplement: 43
- BibTeX: 40
- EndNote: 54
Viewed (geographical distribution)
| Country | # | Views | % |
|---|
| Total: | 0 |
| HTML: | 0 |
| PDF: | 0 |
| XML: | 0 |
- 1
“MeteoSaver v1.0: a machine-learning based software for the transcription of historical weather data” by Derrick et al.
https://doi.org/10.5194/egusphere-2024-3779
Preprint. Discussion started: 10 June 2025
General Assessment
This manuscript presents MeteoSaver v1.0, an open-source, machine-learning based pipeline for the transcription, quality control, and structuring of historical meteorological records. The work is technically strong, well motivated, and relevant for climate data rescue, particularly in data-scarce regions.
The system is a valuable contribution to the field of historical climate data rescue, and the open-source, modular design is commendable.. The paper represents a valuable contribution at the interface of climate science, machine learning, and data engineering.
To strengthen the manuscript, I recommend expanding or more clearly contextualizing the validation, clarifying accuracy requirements for climate applications, and addressing potential biases introduced by rule-based QC. Addressing these points will significantly enhance the practical usefulness of the software.
The manuscript reports several performance metrics (e.g., transcription match rates, MAE, quality flags). The reported median match rate of approximately 74% between MeteoSaver outputs and manual transcription is relatively low for climate data rescue applications, where accuracy requirements are often stringent. While the authors also report a median MAE of 0.3 °C for temperature, the relationship between these metrics and their implications for downstream climate analyses is not sufficiently discussed. The paper should sufficiently explain:
The authors should clearly link their validation results to the types of climate analyses for which MeteoSaver outputs are (or are not) suitable, and explicitly discuss limitations.
I recommend “minor revision”.
Below are few Minor Comments
p.14: “Following the transcription of the data, quality assessment and quality control (QA/QC) is carried out to ensure the final output data is highly accurate with reference to the original handwritten daily temperature records (see Fig. 9).”
>> The phrase “highly accurate” is not operationally defined. It would strengthen the methodology to clarify whether “accuracy” here refers to:
Clarifying this will help readers understand what the QA/QC module is designed to guarantee.
p.16:
“If this condition is not met, a specific adjustment, unique to our sheets, is applied: the first digit is removed from the value, and the cell is flagged to indicate this manipulation (see Fig. 11 a-b, with manipulated values in b shown in orange).”
>> This is a data transformation rule, not only a quality check. It would help to explicitly describe this as a correction operation and to specify its assumptions (e.g., why the first digit is assumed to be erroneous, and under what conditions this may fail).
“However, if the check is passed, the transcribed temperature values are then adjusted to match the required decimal places, set to one in this case (see Fig. 11 b–c).”
>> This step modifies the data but is not mathematically described. Please clarify:
“For the daily maximum temperature threshold, we use 40°C. For the daily minimum temperature threshold, we use 5°C.”
>> The manuscript would benefit from a brief discussion of how sensitive the results are to these fixed thresholds, and whether they are intended to be region-specific or globally applicable.
p.19:
“Only the confirmed (green) daily temperature values are passed to the next module, Data Formatting and Upload (sect. 3.6).”
>> This implies that a large portion of transcribed data may be excluded. Please indicate the proportion of discarded values and discuss potential impacts on time series completeness. Here the manuscript transitions from checking to correcting. Explicitly distinguishing these two roles would improve conceptual clarity.
“Two examples … illustrate the sequence of QA/QC checks performed on the initial transcribed values, leading to the final confirmed values (flagged in green).”
>> Figure 11 shows the propagation of flags and value states, but the underlying equations and replacement rules are not visible in the figure. Consider annotating the panels with the rule names (threshold, digit removal, Eq. 1-4, etc.) to make the logic traceable.
p.20:
“At this stage, an additional check is performed, which was not included in the QA/QC module due to the availability of longer temperature series at this point.”
>> This introduces a new methodological step after the main QA/QC description. For structural clarity, it may be preferable to describe this earlier as an optional extension of Module 5.