Python-Fortran Hybrid Programming for Deep Incorporation of AI and Physics Modeling and Data Assimilation (Hf2pMDA_1.0)

Abstract. Artificial intelligence (AI) provides an unprecedented opportunity for advancing physics numerical modeling including data assimilation, which is a high-efficient and critically-important tool for advancing our understanding on Earth system and its applications. At the same time, deep incorporation of AI and physical modeling can make great driving to advance AI by injecting it rich physics from long time physics-based modeling development. However, since such physics models are conventionally coded in Fortran and AI algorithms usually are conveniently designed in Python, difficulties exist to directly incorporate AI algorithms into physics models, vice versa. Here, based on a f2py protocol, we have developed a procedure that implements an infrastructure which conveniently conducts Python and Fortran hybrid modeling and data assimilation (Hf2pMDA) to form a program entity so that AI algorithms and physical models can invoke mutually. As examples, within Hf2pMDA, a climate weakly coupled data assimilation (WCDA) system is naturally upgraded to a strongly CDA (SCDA) system, and a 1 km high-resolution weather DA system is conveniently implemented within a multi-layer downscaling model that has multiscale DA in different nesting layers. In the climate SCDA system, a coupled general circulation model (CGCM) and multiscale filtering algorithm is integrated by a Python main controller (PMC) that calls Fortran CGCM components and WCDA modules as well as a data-trained SCDA algorithm by latent space variational autoencoder (VAE) in Python. In the high-resolution weather DA system, the downscaled model consisting of traditional Fortran DA modules in all mother domains and Python VAE DA algorithm in the central child domain is integrated by a PMC that organizes these components. With convenient realization of deep incorporation of any AI algorithm and physics model, the Hf2pMDA has a great potential to make progresses on both AI and scientific modeling.