the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
Improving heat transfer predictions in heterogeneous riparian zones using transfer learning techniques
Abstract. Data-driven deep learning models usually perform well in terms of improving computational efficiency for predicting heat transfer processes in heterogeneous riparian zones. However, traditional deep learning models often suffer from accuracy when data availability is limited. In this study, a novel deep transfer learning (DTL) approach is proposed to improve the accuracy of spatiotemporal temperature distribution predictions. The proposed DTL model integrates the physical mechanisms described by an analytical model into the standard Deep Neural Networks (DNN) model using a transfer learning technique. To test the robustness of the proposed DTL model, the influence of the number of observation points at different locations, streambed heterogeneity (𝜎²lnK =0, 0.2, 0.5, and 1.0), and observation noise levels (𝜎𝑁𝑜𝑖𝑠𝑒 =0.025, 0.05, 0.075) on the MSE values between the observed and predicted temperature fields. Results indicate that the DTL model significantly outperforms the DNN model in scenarios with scarce training data, and the mean MSE values decrease with increasing observation points for both DTL and DNN models. The mean MSE values for both the DTL and DNN models approach zero as the number of observation points increases to 200, indicating that both DTL and DNN models perform satisfactorily. Furthermore, increasing 𝜎²lnK and 𝜎𝑁𝑜𝑖𝑠𝑒 raises the mean MSE values of the DTL and DNN models, with the DTL model exhibiting greater robustness than the DNN model, highlighting its potential for practical applications in riparian zone management.
- Preprint
(2018 KB) - Metadata XML
-
Supplement
(637 KB) - BibTeX
- EndNote
Status: open (until 12 Mar 2025)
-
RC1: 'Comment on egusphere-2024-4145', Anonymous Referee #1, 06 Mar 2025
reply
This manuscript proposes a Deep Transfer Learning (DTL) approach to improve the accuracy of spatiotemporal temperature distribution predictions in heterogeneous riparian zones. Using transfer learning, the authors integrate analytical solution outputs for a homogeneous medium into a Deep Neural Network (DNN) and employ a 2D numerical model output for a heterogeneous medium as their synthetic data. They tested their approach by comparing the DTL to a DNN trained solely on synthetic data across various heterogeneous media and noise levels. Their findings indicate that the DTL model outperforms the DNN model in scenarios with limited training data and demonstrates greater robustness to data noise, which may have practical applications in riparian zone management.
The current version of the manuscript requires significant work. Essential information regarding the physical-based models used to train the DTL and DNNs is missing, as well as clarifications on the input and output variables of the machine learning models needed for testing and reproducing the work presented. Additionally, the authors should include the reasoning behind their sampling criteria and how it is linked to the physical process they are modeling, as well as highlight how their novel framework differs or adds from work done by previous authors. With the latter in mind, I cannot accept the manuscript in its current form.
Below, I have listed comments and suggestions, hoping they may help improve the manuscript’s quality.
Specific Comments
- The physics-based models need further clarification.
- The authors based their analytical and numerical models on previous work performed by Shi et al. (2023) and they present some of the equations and boundary conditions in the manuscript and the supplementary information. However, the manuscript does not clarify the actual domain of the system. Are they using the model's domain as the conceptual model presented in Figure 1? If so, why are the modeling results presented in a square? Is this an inset of the larger domain? If so, where is the inset located for the whole model? If it is not an inset, is the domain different from the one presented by Shi et al. (2023)? If so, why is its extent shorter than that of the original study? A clear description of the conceptual model and its boundary conditions should be included in the main manuscript to aid in the understanding of the physical process.
- Additionally, the groundwater flow model and its boundary conditions are not mentioned. Is this the same model as the one used in Shi et al. (2023)? This should be included and clarified in the manuscript for an integral understanding of the process that the data-driven models are trying to reproduce.
- Incidentally, part of the work looks into heterogeneity, and the authors present their heterogeneous fields. Nonetheless, there is no mention of which hydraulic conductivity value is used for the homogeneous case. The authors only mention variations in the Darcy’s fluxes (qx and qz) in line 167. How are these fluxes calculated? What values are used for head gradients? Are the variations of these Darcy’s fluxes related to boundary conditions or fluxes through the domain? I suggest including the Darcy flux equation and leaving the variations only to hydraulic conductivity to be consistent with the heterogeneous cases.
- The authors only present the fields for hydraulic conductivity and absolute errors, and there is no plot of the temperature field they are trying to reproduce. Are these fields different from each other? How does the heterogeneous domain affect the temperature distributions? I suggest adding a figure with the temperature fields for the analytical and the numerical solutions so that the reader can understand how these fields vary throughout the domain and what the data-driven models are missing.
- With respect to the machine learning models
- The authors mention in line 15 that this work “[proposes] a novel Deep Transfer Learning (DTL) approach […] to improve the accuracy of spatiotemporal temperature distribution predictions.” However, a similar approach has been explored in Zhang et al. (2023) for the prediction of hydraulic heads in heterogeneous aquifers. The authors should clearly specify the improvements or modifications made to the framework compared to Zhang et al. (2023), beyond the difference in application.
- In line 222, the authors mention that they restricted the number of epochs in the model training. Is there a reason why these models cannot be trained with different epochs until they reach the same convergence? Also, what about the other hyperparameters of the DNN models (i.e., number of nodes, number of layers, epochs, and activation functions, among others), have the authors considered testing a range of these parameters to get the best set of DNN?
- Part of using these data-driven approaches is leveraging the current available data to predict variables that are difficult, expensive, or impractical to measure. With this in mind, the authors should be clear about what variables they are using as input to predict the temperature fields. Are they using the hydraulic heads and temperature of the stream? Are they using variables related to the geology of the site? Or are they using temperature data from previous timesteps? All of this is important because if we were to use these models to predict the temperature in a given field site, we would need to know what variables we should measure to be able to have an accurate prediction.
- Furthermore, the authors should link their sampling criteria to the physical process they are trying to reproduce with data-driven approaches. For instance, grabbing more than 50 samples in a 1-meter cross-section with some spaced less than 0.1 meters horizontally is impractical and inefficient. I suggest the authors approach the sampling criteria as they were placed in the field, and are tasked to maximize the location of their thermistors or other measuring devices. This reviewer believes this approach can benefit the scientific community and add value to the manuscript.
- Consider including an additional paragraph or sentences that describe other approaches to create physics-informed machine learning models (e.g., Arcomano et al., 2022; M. Raissi et al., 2019; Maziar Raissi & Karniadakis, 2018; Yeung et al., 2022).
- I suggest the authors add more information in the discussion section. Where they highlight the importance of their work and how it relates to other approaches. I suggest also highlighting the transferability of this framework to other settings, as well as things that scientists should take into account.
Technical Corrections
Besides the comments described above, I have a few technical recommendations for the manuscript.
- The manuscript has multiple sentences that are difficult to read or have grammatical errors. Among them are:
- Lines 17-20 are difficult to read and contain variables that are not previously defined
- The sentence in lines 22-23 is redundant, so consider removing it.
- Grammar in line 89 “Newly proposed demonstrates”
- In line 294 should be “centers” instead of “centres”
- What do you mean by “it is postulated that the thermal and hydraulic properties of the streambed maintain uniformity”? (Lines 98-99). Are you referring to the fact that these variables remain constant throughout the simulation? Please clarify
- It should be “no heat flux boundary” in line 102.
- I recommend collapsing equations (1a) through (1c) to a single equation with a subscript i that is later described.
- Line 144 states that “The hyperparameters θT for the fine-tuning model is acquired through the optimization of the loss function delineated by…” By definition, a hyperparameter cannot be estimated with model training. They are set by the user. I think that you mean “The parameters” instead of “The hyperparameters.”
- Some variables, such as qx and qz, are not defined in the main manuscript. Since the manuscript should be self-contained, these variables should be specified in the text.
- Remember to add the units of the Mean Square Error (MSE) values.
- The text in Figures 5, 6, and 10 is difficult to read. Consider increasing the fonts. Also, include the units of the variables plotted.
- Consider using the same y-scale for Figures 7, 9, and 11. This would aid in the comparison.
References
Arcomano, T., Szunyogh, I., Wikner, A., Pathak, J., Hunt, B. R., & Ott, E. (2022). A Hybrid Approach to Atmospheric Modeling That Combines Machine Learning With a Physics-Based Numerical Model. Journal of Advances in Modeling Earth Systems, 14(3), e2021MS002712. https://doi.org/10.1029/2021MS002712
Raissi, M., Perdikaris, P., & Karniadakis, G. E. (2019). Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics, 378, 686–707. https://doi.org/10.1016/j.jcp.2018.10.045
Raissi, Maziar, & Karniadakis, G. E. (2018). Hidden physics models: Machine learning of nonlinear partial differential equations. Journal of Computational Physics, 357, 125–141. https://doi.org/10.1016/j.jcp.2017.11.039
Shi, W., Zhan, H., Wang, Q., & Xie, X. (2023). A Two-Dimensional Closed-Form Analytical Solution for Heat Transport With Nonvertical Flow in Riparian Zones. Water Resources Research, 59(8), e2022WR034059. https://doi.org/10.1029/2022WR034059
Yeung, Y.-H., Barajas-Solano, D. A., & Tartakovsky, A. M. (2022). Physics-Informed Machine Learning Method for Large-Scale Data Assimilation Problems. Water Resources Research, 58(5), e2021WR031023. https://doi.org/10.1029/2021WR031023
Zhang, J., Liang, X., Zeng, L., Chen, X., Ma, E., Zhou, Y., & Zhang, Y.-K. (2023). Deep transfer learning for groundwater flow in heterogeneous aquifers using a simple analytical model. Journal of Hydrology, 626,130293. https://doi.org/10.1016/j.jhydrol.2023.130293
Citation: https://doi.org/10.5194/egusphere-2024-4145-RC1 - The physics-based models need further clarification.
Model code and software
Python codes of the DTL and DNN models Aohan Jin https://github.com/Ahjin-CUG/TL
Viewed
HTML | XML | Total | Supplement | BibTeX | EndNote | |
---|---|---|---|---|---|---|
78 | 22 | 4 | 104 | 24 | 3 | 3 |
- HTML: 78
- PDF: 22
- XML: 4
- Total: 104
- Supplement: 24
- BibTeX: 3
- EndNote: 3
Viewed (geographical distribution)
Country | # | Views | % |
---|
Total: | 0 |
HTML: | 0 |
PDF: | 0 |
XML: | 0 |
- 1