Deep learning tool: Reconstruction of long missing climate data based on multilayer perceptron (MLP)

Yan, Zhang; Tianxin, Xu; Chenjia, Zhang; Daokun, Ma

doi:https://doi.org/10.5194/egusphere-2024-439

Preprints

https://doi.org/10.5194/egusphere-2024-439

Preprints

07 Mar 2024

| 07 Mar 2024

Status: this preprint is open for discussion and under review for Atmospheric Measurement Techniques (AMT).

Deep learning tool: Reconstruction of long missing climate data based on multilayer perceptron (MLP)

Zhang Yan, Xu Tianxin, Zhang Chenjia, and Ma Daokun

Abstract. Long-term monitoring of climate data is significant for grasping the law and development trend of climate change and guaranteeing food security. However, some weather stations lack monitoring data for even decades. In this study, 62 years of historical monitoring data from 105 weather stations in Xinjiang were used for missing sequence prediction, validating proposed data reconstruction tool. First of all, study area was divided into three parts according to the climatic characteristics and geographical locations. A deep learning tool based on multilayer perceptron (MLP) was established to reconstruct meteorological data with three time scales (Short term, cycle and long term) and one spatio dimension as inputing, filling in long sequence blank data. By designing an end-to-end model to autonomously detect the locations of missing data and make rolling predictions, we obtained complete meteorological monitoring data of Xinjiang from 1961 to 2022. Seven kinds of parameter reconstructed include maximum temperature (Max_T), minimum temperature (Min_T), mean temperature (Ave _ T), average water vapor pressure (Ave _ WVP), relative humidity (Ave _ RH), average wind speed (10 m Ave _ WS), and sunshine duration (Sun_H). The quality of reconstructed data was evaluated by calculating correlation coefficient with the monitored sequences of nearest station. Results show that, proposed model reached satisfied average correlation coefficient for Max_T, Min_T, Ave _ T and Ave _ WVP parameters are 0.969, 0.961, 0.971 and 0.942 respectively. The average correlation coefficient of Sun_H and Ave _ RH are 0.720 and 0.789. Although it is difficult to predict extreme values, it can still capture the period and trend; the reconstruction effect of 10 m Ave _ WS is poor, with the average similarity of 0.488. Finally, we published the trained parameter files and prediction codes as a micro service on the Agricultural Smart Brain platform, which provides firstly a deep learning tool for rapid and reliable reconstruction of meteorological monitoring data.

Received: 14 Feb 2024 – Discussion started: 07 Mar 2024

Download & links

Zhang Yan, Xu Tianxin, Zhang Chenjia, and Ma Daokun

Status: open (until 24 May 2024)

Post a comment Subscribe to comment alert

RC1:
'Comment on egusphere-2024-439', Anonymous Referee #1, 20 Mar 2024 reply
This study used multi-layer perceptrons (MLP) to reconstruct missing climate data series by a deep learning approach which provided a useful tool under limiting data source. The manuscript is well-written and developed a practical and helpful methodology. My main concerns are:
the manuscript requires clear section of method and materials; which data are used to train the model and which data is for validation, and how to do the training and validation.

requires discussion with limitations, implications. The reliability of MLP in rapidly reconstructing meteorological data is mentioned, but it is not discussed the applicability of the model in different environments or datasets.

conclusion needs to be more general and to address how can use the results (tool) to help science and production.

It is mentioned that the model was designed with four MLP modules and two fully connected predictive heads, but it is not explained why this specific model structure was chosen and how the performance of the model is ensured.

Discuss why there are low correlation coefficients for sunshine and wind speed.

I suggest to restructure the manuscript following the logic of, introduction, method and materials, results, discussions, and conclusions.

Detail comments:
L10, 16: add space between two words. Also check whole text.
L14-16, delete the expression of evaluation, just give the results.
L16, results showed that….,
In the abstract, it is better to give how many data the paper used and how many data (%) were missed in the data sheet. The parameter for evaluation, a root mean square error (RMSE) might be help to understand the accuracy of the reconstruction.
Keywords: use different terms with title
L26-36, too long to say the importance of climate change, not closely relevant to this study, just general statement is enough.
L86: only 143 missing data? Give total data and missing data percentage. This would show the power of method performance.
L85-95: here mostly say how to do, but need to say the clear objectives of the paper. How to do could be move to the M&M section.
L96-123: This section gives the information of study area and data from, but too vague, for example, table is not self-clarification. The section could be belong to 2. M&M, 2.1 Data source. In fact, where is the study done is not so important but what is study data is very important. Suggest to develop a table (combine figure 1, table 1 and figure 2) including name of site, latitude, observed yeas, total data, missing data percentage. Also clarify which data is used for training the model and which data were used to validate the model.
L123- : this section describes the model, belonging to M&M

Reply
Citation: https://doi.org/10.5194/egusphere-2024-439-RC1

Zhang Yan, Xu Tianxin, Zhang Chenjia, and Ma Daokun

Viewed

Total article views: 147 (including HTML, PDF, and XML)

HTML	PDF	XML	Total	BibTeX	EndNote
110	26	11	147	8	9

HTML: 110
PDF: 26
XML: 11
Total: 147
BibTeX: 8
EndNote: 9

Views and downloads (calculated since 07 Mar 2024)

Month	HTML	PDF	XML	Total
Mar 2024	89	23	7	119
Apr 2024	21	3	4	28

Cumulative views and downloads (calculated since 07 Mar 2024)

Month	HTML	PDF	XML	Total
Mar 2024	89	23	7	119
Apr 2024	21	3	4	28

Viewed (geographical distribution)

Total article views: 169 (including HTML, PDF, and XML) Thereof 169 with geography defined and 0 with unknown origin.

Country	#	Views	%

Latest update: 28 Apr 2024

Short summary

A deep learning tool based on multilayer perceptron (MLP) is established for the meteorological data reconstruction at three time scales. Seven parameter reconstruction methods were used to validate the proposed data reconstruction tool. Finally, the trained parameter files and prediction code were released as microservices on the Agricultural Smart Brain platform, which provides the first deep learning tool for rapid and reliable reconstruction of meteorological monitoring data.


Total:	0
HTML:	0
PDF:	0
XML:	0