Using machine learning algorithm to retrieve cloud fraction based on FY-4A AGRI observations

Abstract. Cloud fraction as a vital component of meteorological satellite products plays an essential role in environmental monitoring, disaster detection, climate analysis and other research areas. A long short-term memory (LSTM) machine learning algorithm is used in this paper to retrieve the cloud fraction of AGRI (Advanced Geosynchronous Radiation Imager) onboard FY-4A satellite based on its full-disc level-1 radiance observation. Correction has been made subsequently to the retrieved cloud fraction in areas where solar glint occurs using a correction curve fitted with sun-glint angle as weight. The algorithm includes two steps: the cloud detection is conducted firstly for each AGRI field of view to identify whether it is clear sky, partial cloud or overcast cloud coverage within the observation field. Then the cloud fraction is retrieved for the scene identified as partly cloudy. The 2B-CLDCLASS-LIDAR cloud fraction product from Cloudsat& CALIPSO active remote sensing satellite is employed as the truth to assess the accuracy of the retrieval algorithm. Comparison with the operational AGRI level 2 cloud fraction product is also conducted at the same time. During daytime, the probability of detection (POD) for clear sky, partly cloudy, and overcast scenes in the official operational cloud detection product were 0.5359, 0.7041, and 0.7826, respectively. The POD for cloud detection using the LSTM algorithm were 0.8294, 0.7223, and 0.8435. While the operational product often misclassified clear sky scenes as cloudy, the LSTM algorithm improved the discrimination of clear sky scenes, albeit with a higher false alarm rate compared to the operational product. For partly cloudy scenes, the mean error (ME) and root-mean-square error (RMSE) of the operational product were 0.2374 and 0.3269. The LSTM algorithm exhibited lower ME (0.1134) and RMSE (0.1897) than the operational product. The large reflectance in the sun-glint region resulted in significant cloud fraction retrieval errors using the LSTM algorithm. However, after applying the correction, the accuracy of cloud cover retrieval in this region greatly improved. During nighttime, the LSTM model demonstrated improved POD for clear sky and partly cloudy scenes compared to the operational product, while maintaining a similar POD value for overcast scenes and a lower false alarm rate. For partly cloudy scenes at night, the operational product exhibited a positive mean error, indicating an overestimation of cloud cover, whereas the LSTM model showed a negative mean error, indicating an underestimation of cloud cover. The LSTM model also exhibited a lower RMSE compared to the operational product.