the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
| 07 Oct 2024
Multi-hazard susceptibility mapping in the karst context using a machine-learning method (MaxEnt)
Abstract. In this study, we extend the application of the Maximum Entropy model (MaxEnt), traditionally applied to ecological research and less explored in natural hazard studies, to a novel context by characterising a multi-hazard scenario (i.e., flood-triggered sinkholes) in the Orléans karst region (Val d'Orléans) of France. Many regions of the world exhibit complex hazard landscapes where networks of multi-hazard interrelationships (cascades) pose challenges due to the potential interactions between hazards and the different temporal and spatial scales of hazard events. While mountainous, coastal and volcanic regions have been recognised as multi-hazard forming zones, karst terrains have received little attention despite being prone to multi-hazard events due to their distinct geology, geomorphology, hydrogeology and other environmental characteristics. Incorporating karst-specific multi-hazard scenarios supports disaster risk reduction efforts by raising the awareness of citizens, protecting elements at risk and facilitating decisions on disaster prevention. To support this aim, we developed a multi-hazard susceptibility map for the karst region of Val d'Orléans that characterises flood-triggered sinkholes. We applied MaxEnt, a machine learning method, to forecast the spatial probability distribution of flood-triggered sinkholes. Model inputs included the location of past sinkhole occurrences and geo-environmental factors contributing to sinkhole formation (e.g., topography, local geology, hydrology and flood hazard). We validated the performance of the model by initially using 70 % of the sinkhole inventory data and keeping the remaining 30 % for testing. This validation process assessed the model's performance using the Area Under the Curve of the Receiver Operating Characteristic (AUC-ROC). The resulting map reveals areas located up to 1 km south of the Loire River and areas with lowest elevation with highest susceptibility to flood-triggered sinkholes. We conclude that our approach to producing this type of multi-hazard scenario and map is useful for identifying flood-triggered sinkholes in Val d'Orléans and other karst areas around the globe, supporting effective land use planning.
Competing interests: SF and SM are members of the editorial board of Natural Hazards and Earth System Sciences.
Publisher's note: Copernicus Publications remains neutral with regard to jurisdictional claims made in the text, published maps, institutional affiliations, or any other geographical representation in this paper. While Copernicus Publications makes every effort to include appropriate place names, the final responsibility lies with the authors. Views expressed in the text are those of the authors and do not necessarily reflect the views of the publisher.- Preprint
(2632 KB) - Metadata XML
- BibTeX
- EndNote
Status: final response (author comments only)
-
RC1: 'Comment on egusphere-2024-1779', Anonymous Referee #1, 18 Jun 2025
-
AC1: 'Reply on RC1', Hedieh Soltanpour, 16 Jan 2026
Dear reviewer,
We greatly appreciate the time and effort you took to evaluate our work and for highlighting both its strengths and areas for improvement. We are also grateful for your positive remarks regarding the scientific significance, methodological clarity, and relevance of our study.
We are currently revising the manuscript and will carefully address all the points you raised. In particular:
1) Groundwater levels and Fig. 6e: Thank you for pointing out the inconsistency in the labelling and interpretation. We admit that by groundwater level, we meant depth to groundwater. Therefore, we will modify the terminology and the figure caption accordingly.
2) Writing style and tense: We are reviewing the manuscript to improve consistency in tone and style. The Methods section will be revised to consistently use the past tense where appropriate.
3) References: We will check all references and in-text citations to correct inconsistencies, add missing citations, and fix any misspellings.
4) Minor typos and language improvements: We will incorporate your suggested edits to improve clarity and precision, and refine phrasing throughout the text as you noted.
5) We will make sure to revise the title as suggested and will improve the figure captions and explanatory text to clarify groundwater conditions and ensure consistency in terminology.
We sincerely appreciate your helpful feedback and are confident that these revisions will strengthen the manuscript. A full, point-by-point response will accompany the revised submission.
Citation: https://doi.org/10.5194/egusphere-2024-1779-AC1
-
AC1: 'Reply on RC1', Hedieh Soltanpour, 16 Jan 2026
-
RC2: 'Comment on egusphere-2024-1779', Andreas Wunsch, 17 Dec 2025
In their article „Multi-hazard susceptibility mapping in the karst context using a machine-learning method (MaxEnt)“, Soltanpour et al. apply the Maximum Entropy model (MaxEnt) to characterize flood-triggered sinkholes in the Val d'Orléans karst region of France. The authors argue convincingly that karst terrains are underrepresented in the multi-hazard literature despite their inherent potential for hazard interactions. This topic is of clear interest and relevance to the readers of NHESS, and the approach of integrating official flood hazard maps as a contributing factor for sinkhole susceptibility mapping represents a meaningful methodological contribution. The manuscript is well-written, logically structured, and demonstrates careful conceptualization. I did enjoy reading it. In light of the comments below, I recommend publication after major revisions. My main criticism is that the authors tend to overstate certain conclusions and claims throughout the manuscript, particularly regarding the interpretability of their model outputs and the validation of the model itself. I encourage the authors to adopt a more cautious tone in several places, which would strengthen the credibility of their findings.
As a disclosure, while I have sound knowledge of karst systems and profound experience with ML applications in general, I have not personally applied the MaxEnt algorithm nor conducted hazard susceptibility assessments; my comments on these specific aspects are therefore to the best of my knowledge. Please also note, that I used assistance from Claude Opus 4.5 to structure my thoughts on the manuscript, generate text for this review and to identify minor corrections (wording, commas etc.). I bear full responsibility for the content of this review myself.
General Assessment:
The authors identify elevation as the dominant predictor for flood-triggered sinkhole susceptibility (jackknife test, response curves). However, given the limited elevation range in the study area (82–126 m) and the consistent east-to-west decrease in elevation, I am concerned that elevation may serve as a spatial proxy rather than a causally meaningful predictor. The model may effectively learn „location within the study area“ rather than a true elevation-sinkhole relationship. I hypothesize that similar results could be obtained using distance-to-boundary or other location-describing features. This does not invalidate the susceptibility map per se, but it undermines the interpretability of conclusions regarding elevation as a driving factor. I strongly encourage the authors to discuss this limitation and alter statements attributing causal importance to elevation. In general, I would like to encourage authors to be careful of where causality can be inferred from the analyses and where it cannot. In my view, the data hardly allow any conclusions to be drawn about causality, but rather about presumed connections or correlations.
A similar concern applies to the groundwater response curve, where probability declines sharply beyond 15.5 m. To me, there is no obvious hydrogeological mechanism that would explain why erosion and dissolution processes would cease at higher groundwater levels. Could groundwater level act as a proxy for another underlying factor (e.g., subsurface structure, aquifer properties) that is not explicitly included in the model? Please discuss.
The authors state that one objective is „to evaluate the potential for the MaxEnt model to be used in multi-hazard mapping“ (Objective 3). While the application is successfully demonstrated, a true evaluation of MaxEnt's potential would require comparison with an established baseline method. The authors themselves cite Perrin et al. (2015), who applied weight-of-evidence in the same study area. Why not include this as a benchmark? Without such comparison, Objective 3 remains only partially addressed.
Section 4.1 „Model Validation“ is misleadingly titled. I would argue it is more of an evaluation than a validation of the model. The statement that „validation data includes both training and testing datasets“ (Line 529-530) is inconsistent with standard ML terminology and contradicts Line 492-495. Please clarify the evaluation and validation strategy in general. I recommend renaming this section (e.g., „Model Performance Assessment“) and clarifying the data split procedure. Please also explain whether AUC was calculated on the test set only (30%) or on combined data.
Specific Comments:
Lines 63-79: Consider to remove or substantially shortening this part. A general elaboration on multi-hazards is not beneficial for the readability of this study. If you keep these paragraphs, use metric units (not inches).
Objective 3 – Baseline comparison (Lines 147-149): The authors state that one objective is „to evaluate the potential for the MaxEnt model to be used in multi-hazard mapping.“ While the application is successfully demonstrated, a rigorous evaluation would benefit from comparison with an established baseline method. The authors cite Perrin et al. (2015), who applied weight-of-evidence in the same study area and produced a susceptibility map. Why not compare MaxEnt performance against this existing approach? Without such comparison, Objective 3 remains only partially addressed, and claims about MaxEnt's suitability for multi-hazard mapping are difficult to substantiate.
Figure 1 (p. 6): I don't see any bidirectional arrows as indicated. I also think that some of the labels in the soil refer more to the limestone/bedrock and are therefore in the wrong place. Karst should also be better marked and labeled to make it clear that this is where the actual dissolution processes take place.
Lines 492-495 (Data splitting): The authors use a 70/30 random split for training and testing. However, given the strong spatial clustering of sinkholes along the Loire River (58% within 1 km, Figure 4), random splitting may result in spatially proximate sinkholes appearing in both training and test sets. This spatial autocorrelation can inflate AUC values. Did the authors consider spatial blocking or leave-one-cluster-out cross-validation? I recommend acknowledging this limitation.
Section 4.1 „Model Validation“ (Lines 519-533): This section title is a bit misleading. True validation would require independent ground truth data (e.g., sinkholes that occurred after model training, or expert-validated susceptibility zones). The statement that „validation data includes both training and testing datasets“ (Line 529-530) is confusing and contradicts the earlier description of the 70/30 split (Lines 492-495). I recommend:
- Renaming this section to „Model Performance Assessment“ or „Model Evaluation”
- Clarifying whether AUC was calculated on the test set only (30%) or on combined data.
- As I understand AUC in MaxEnt measures discrimination between presence locations and background points (pseudo-absences), not true presence-absence classification. Please include and discuss this aspect.
Lines 584-593 (Elevation response curve interpretation): This is one of my main concerns. The authors identify elevation as the dominant predictor and interpret the optimal range (90–105 m) as causally meaningful for flood-triggered sinkhole susceptibility. However, I am skeptical of this interpretation for the following reasons:
- The study area has very limited elevation variation (82–126 m, i.e., only ~44 m range).
- Elevation decreases consistently from east to west across the study area (as noted in Lines 394-395).
- There is no compelling hydrogeological mechanism explaining why sinkholes should occur preferentially at 90–105 m elevation per se.
I hypothesize that elevation serves as a spatial proxy for location within the study area rather than a causally meaningful predictor. The model may effectively learn „areas in the central-western part of the valley“ rather than a true elevation-sinkhole relationship. I would expect similar model performance if elevation were replaced by „distance to eastern boundary“ or similar location-describing features.
This does not invalidate the susceptibility map, but it undermines the interpretability of conclusions attributing causal importance to elevation. I strongly encourage the authors to:
- Discuss this limitation explicitly.
- Temper statements such as „Elevation appears to be the main variable“ (Line 714) and similar claims throughout the manuscript.
- Consider testing whether a location-based variable (e.g., X-coordinate or distance to a reference point) yields comparable model performance.
Lines 609-615 (Groundwater response curve interpretation): The authors state that sinkhole probability „swiftly declines“ beyond 15.5 m groundwater level. This interpretation requires more critical discussion. From a hydrogeological perspective, there is no obvious mechanism that would cause erosion and dissolution processes to cease at higher groundwater levels. If anything, higher hydraulic gradients associated with elevated groundwater might be expected to increase erosion potential.
Could GWL serve as a proxy for another factor not explicitly included in the model (e.g., aquifer properties, subsurface geology, or proximity to specific hydrogeological features)? I encourage the authors to discuss this possibility and avoid over-interpreting the response curve as reflecting a direct causal relationship.
Lines 603-609 (Alluvial thickness response curve): The response curve shows very low sinkhole probability where alluvial thickness is minimal (close to 0). The authors interpret this as consistent with the suffosion mechanism, which requires cover material. While this interpretation is reasonable for cover-collapse sinkholes, I would appreciate a brief discussion of whether solution sinkholes (without cover) could still occur in areas with minimal alluvium. Are there exposed karst features in the study area that might represent a different sinkhole type not captured by this model?
Lines 658-667 & Figure 9 (Jackknife test results): The jackknife test reveals that "Flood hazard zones" is among the weakest predictors when used in isolation (Figure 9). This finding appears to contradict the central premise of the paper that flooding is a key trigger for sinkhole formation. If flood hazard contributes minimally to the model's predictive power compared to elevation, groundwater, and alluvium thickness, what does this imply for the "multi-hazard" framing? (Be careful with causality here, as elaborated above). The authors acknowledge this partially (Lines 725-731), attributing it to spatial overlap with other variables. However, I encourage a more explicit discussion of whether this undermines the "multi-hazard" claim or whether it simply reflects that flood hazard is already captured by correlated variables.
Lines 531-533 & 711-713 (Model performance vs. conclusions): The authors correctly characterize the AUC of 0.702 as "satisfactory" (Line 531), which indicates moderate discrimination ability. However, some statements in the manuscript appear to overstate the model's reliability (e.g., "the model forecasts the highest relative probability," Line 586-587; "Elevation appears to be the main variable," Line 714). I recommend that the authors temper these statements to align with the moderate predictive performance. Phrases such as "the model suggests" or "results indicate a tendency" would be more appropriate than definitive attributions.
Minor Comments:
Figure 3: add “Arc” to GIS Pro
Figure 5: (e) is it relative or average depth? I assume the latter. Please correct.
Figure 9 (p. 24): The color scheme (red, green, blue) makes the bars difficult to distinguish, particularly for readers with color vision deficiencies, but in its current form also for those without deficiencies. Please consider recoloring the bars.
Figure 10 (p. 26): The colors in the figure do not match the description in the text. Please verify and correct the color assignments in either the figure or the caption/text.
Line 64: “(Category 1)unleased“ → missing space before „unleashed“
Line 75: “800 houses(de Ruiter“ → missing space before citation
Line 111: Missing end of sentence: „…sciences. Bianchin …”
Citation: https://doi.org/10.5194/egusphere-2024-1779-RC2 -
AC2: 'Reply on RC2', Hedieh Soltanpour, 16 Jan 2026
Dear Dr. Andreas Wunsch,
We are very grateful for your careful review and for the insightful and constructive comments on our manuscript. We appreciate your recognition of the contribution of this work to both methodological practice and the multi-hazard literature in karst terrains.
Your comments raise important and well-founded points that will help us clarify key aspects of the methodology, interpretation, and framing of the study. We thank you for each one of the comments you thoughtfully provided. To avoid repetition, we respond below to each comment in turn and describe the revisions we will make accordingly.
Before responding to individual comments, we would like to agree with the reviewer’s concern regarding potential over-interpretation of model outputs. As MaxEnt is a presence-only, pattern-based method that captures statistical associations rather than causal mechanisms, we will carefully revise the manuscript to ensure that interpretations remain descriptive and consistent with this methodological scope. Process-based explanations will be explicitly framed as plausibility checks grounded in existing geo-environmental knowledge, and the tone will be adjusted accordingly.
1) Elevation Concern (a spatial proxy rather than being causally meaningful): Given the relatively limited spatial extent of the study area, the influence of spatial autocorrelation and the use of variables (predictors) acting as spatial proxies cannot be entirely eliminated. As the primary objective of this study is to produce a susceptibility map identifying areas with different likelihoods of flood-triggered sinkhole occurrence, we fully agree that the data and modelling framework do not allow causal conclusions to be drawn. We will therefore modify the manuscript to avoid causal language and to explicitly acknowledge this limitation, emphasising that the results describe associations and presumed connections rather than mechanistic relationships.
2) Groundwater: We acknowledge that similar concern could arise for the groundwater variable and clarify that groundwater level in this study is not interpreted as a direct causal trigger of sinkhole collapse. In the Val d’Orléans floodplain, groundwater level integrates multiple hydrogeological characteristics - such as aquifer connectivity, conduit development, and interaction with the Loire River- that are difficult to represent explicitly at the regional scale. Its importance in the model therefore reflects its role as an indicator of favourable subsurface conditions for flood-related instability rather than a mechanistic driver. We will revise the manuscript to make this interpretation explicit and to emphasise that the results describe spatial associations within a susceptibility framework, not causal relationships.
3) Objective No 3 (Baseline comparison): While algorithm benchmarking is one way to assess model performance in susceptibility mapping studies, Objective 3 was not intended to evaluate MaxEnt relative to alternative algorithms, but rather to assess its suitability for representing a multi-hazard susceptibility scenario in a karst context, specifically flood-triggered sinkholes. Our evaluation therefore focused on whether MaxEnt can (i) integrate different types of hazard data, including both continuous and categorical variables, (ii) produce stable and interpretable outputs within the data (e.g. response curves and variable importance), and (iii) produce a continuous susceptibility output, allowing fine distinctions in susceptibility between different areas.
Although Weight-of-Evidence represents an established approach for karst susceptibility mapping, it is based on different modelling assumptions and objectives. A direct quantitative comparison with a multivariate, regularised machine-learning framework such as MaxEnt is therefore not straightforward in a multi-hazard context. A direct comparison would require redefining the hazard, using the same predictors and spatial resolution, and aligning modelling assumptions across both approaches. Without this, any differences in results would reflect differences in problem formulation rather than the performance of the algorithms themselves.
We will clarify the wording of Objective 3 in the manuscript to make explicit that the evaluation refers to applicability and interpretability within a multi-hazard framework, rather than algorithmic benchmarking, and we acknowledge that comparative assessments against established methods represent a valuable direction for future work.
4) Model Validation: We acknowledge that in much of the hazard susceptibility and risk-mapping literature, ROC-AUC derived from data splitting has traditionally been referred to as “model validation”. However, we agree that under a stricter interpretation, true validation would require independent or temporally distinct ground-truth data, which are not available for this study.
To avoid ambiguity we will therefore rename this section to “Model Performance Assessment” and will clarify that AUC values are computed using the held-out test data (30%) averaged over 50 replicate runs. We also explicitly note that, in the MaxEnt presence-only framework, AUC quantifies the model’s ability to discriminate presence locations from background points rather than true presence–absence classification. This clarification will be added to improve transparency and to guide interpretation of model performance.
5) Missing bidirectional arrows: The feedback mechanism was intended to be illustrated by the dashed arrows linking (4) secondary flooding back to (2) primary flooding in the Figure 1. However, we agree that the bidirectional feedback is not sufficiently clear in the current version of the figure. We will revise Figure 1 to clearly show this feedback using clearer opposed arrows and improved visual distinction, and update the caption accordingly.
6) Data splitting: We acknowledge that sinkholes in the study area are strongly clustered along the Loire River, which means that a random train-test split cannot fully guarantee spatial independence between the two datasets and may lead to slightly optimistic AUC values. We ran the model 50 times to check that results were stable and not due to a lucky split. The AUC is moderate and consistent, which suggests a real spatial signal rather than overfitting. However, because the data are spatially clustered, we acknowledge that the AUC reflects within-area discrimination rather than fully independent predictive skill.
7) Alluvial thickness response curve: The interpretation of low sinkhole probability at minimal alluvial thickness is indeed based on the dominant suffosion and cover-collapse mechanisms documented in the Val d’Orléans, where sinkhole formation primarily occurs within alluvial deposits overlying the karstified “Beauce” limestone (e.g., Perrin et al., 2015; Noury et al., 2018; Luu et al., 2019).
While solution sinkholes may locally exist in the study area, they are not the predominant sinkhole type captured in the inventory used for model calibration. In the revised manuscript, we will clarify this point and state that the model reflects the sinkhole types most commonly observed in the study area.
8) Flood hazard zones: When considered on its own, the flood hazard variable has limited discriminatory power in a correlative MaxEnt framework, which explains its low jackknife contribution. This does not imply that flooding is irrelevant to sinkhole formation, but rather that its influence is expressed in combination with other conditioning factors. This issue again goes back to the fact that in the Val d’Orléans, flood hazard zones overlaps spatially with other contributing variables (topography, groundwater, geology and alluvium thickness). As a result, much of the spatial signal associated with flood-prone conditions is already captured by these conditioning variables.
This outcome does not question the role of flooding in triggering sinkhole collapses. Flooding acts primarily as a trigger, while the spatial occurrence of sinkholes is largely controlled by predisposing geological and hydrogeological conditions. For this reason, flood hazard alone has limited ability to distinguish sinkhole locations in a static susceptibility model, even though its influence becomes apparent when combined with other controlling factors. We believe that rather than undermining the multi-hazard framing of the study, this result highlights the distinction between triggering factors (flooding) and spatial susceptibility conditions (geo-environmental factors). We will revise the discussion section to clarify this point and to avoid causal interpretation of variable importance.
9) Minor comments: We will modify all the flagged minor comments in the final revised version.
We are again very grateful for this positive, constructive and helpful feedback on our manuscript.Citation: https://doi.org/10.5194/egusphere-2024-1779-AC2
Viewed
| HTML | XML | Total | BibTeX | EndNote | |
|---|---|---|---|---|---|
| 923 | 278 | 157 | 1,358 | 41 | 59 |
- HTML: 923
- PDF: 278
- XML: 157
- Total: 1,358
- BibTeX: 41
- EndNote: 59
Viewed (geographical distribution)
| Country | # | Views | % |
|---|
| Total: | 0 |
| HTML: | 0 |
| PDF: | 0 |
| XML: | 0 |
- 1
Reviewer's comments:
The paper presents the methods adopted and implemented for machine learning of susceptibility mapping in a two component multi-hazard complex, using MaxExtent and on this basis I would suggest that the Scientific significance is Good to excellent - 2-1; the Scientific Quality is good, methods are clearly explained and reproducible, with the exception that the groundwater levels cannot be related to the ground levels (see Fig. 6 e depth to water on the fig and stated as groundwater level in caption, and datum for ground levels is not presented), which may also impact the interpretation and results, therefore my ranking would be Good to Fair 2-3; The Presentation Quality is Good (2) albeit two styles of writing are identifiable in the text, but both are readily interpreted by the reader.
With respect to the presentation there are numerous cases of differences between cited reference dates and those reported in the reference list; not all references cited, e.g. Kim and Nam 2018, Pazzi et al., 2018; line 509 Radosavljevic and Anderson rather than et al.; are in the reference list;
some references need correcting, e.g. Mokhtari or Mokhrarai; .
There are some minor typos/ spelling issues, e.g. line 45 partially = partly, line 53 multi-hazard forming zones = zones that host multi-hazards? Line 80 forming landscape = landscape context? Line 174 the exposure of groundwater to rather than and easily eroded ...; line 178 bottom = base; line 232 Quaternary Alluvium; line 238 could be improved to ... which were mainly attributed to karst collapses ....; line 240 happened = occurred; line 250 for = to; Methods section - should this be written in the past tense? Should the title for section 2 be edited to The multi-hazard environment of karst terrains? Line 591 should flattens out to be falls to? Line 607 should this read with increased thickness of alluvial deposits? Figure 8 e again groundwater conditions need to be made clearer. Line 684 ... means of a classification ...Line 717 groundwater conditions need to be defined more clearly.
The content is relevant for NHESS and its international audience, and the title clearly reflects content albeit the word "the" might sensibly be changed to "a" as karst contexts can be very different. The abstract provides a full and clear overview of the content. Line 23 appears to be missing some words e.g. .... scenarios "in resilience procedures" ... or similar.
The references are relevant and acknowledge prior research. The length is appropriate with a strong discussion and conclusions.
I hope that this helps to progress the paper through the publication process and apologise for the delay and connectivity issues.