the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
How the recursive feature elimination affects the SVM and RF for wildfire modeling? A mountainous case study area
Ali Rezaei Barzani
Parham Pahlavani
Omid Ghorbanzadeh
Pedram Ghamisi
Abstract. This study aims to identify key fire factors via recursive feature elimination (RFE) to generate a forest fire susceptibility map (FSM) using support vector machine (SVM), and random forest (RF) models. The fire zones were derived from MODIS satellite imagery from 2012 to 2017. Further validation of these data has been provided by field surveys and reviews of land records in rangelands and forests; a total of 352 fire points were determined in this study. Seventeen factors involving topography, geomorphology, meteorology, hydrology, and anthropology were identified as being effective primary factors in triggering and spreading fires in the selected mountainous case study area. As a first step, the RFE models of the RF, Extra trees, Gradient boosting, and AdaBoost was used to identify important fire factors among all selected primary factors. The SVM and RF models were applied once on all factors and the second on those derived from RFE models as the key factors in FSM. Training and testing data were divided tenfold, and the model's performance was evaluated using cross-validation (CV). Different metrics were used to measure accuracy, including recall, precision, F1, accuracy, the area under the curve (AUC), Matthews correlation coefficient (MCC), and Kappa. The accuracy assessment process shows that the FSM results are further improved by leveraging RFE models to distinguish the key factors and not include unnecessary factors. The greatest improvement is for SVM, with more than 10.97 % and 8.61 % in the accuracy and AUC metrics, respectively.
Ali Rezaei Barzani et al.
Status: open (extended)
-
CC1: 'Comment on egusphere-2022-1294', Carolina Ojeda, 13 Jan 2023
reply
General comments: This article presented a study case in Iran in which a GIS-type forest fire hazard model is tested. Its novelty relies on the improvement of Fire Susceptibility Maps (FSM) results, which could help GIS experts to improve future analyses for similar topographical conditions.
Specific comments:
1. The subtitle is a bit crowded: "A mountainous case study area". My suggestion here is "A study case in Iran".
2. In the abstract line 15 and section 3.2 the concept "anthropology/anthropological" should be replaced by "human actions/factors" or "anthropic actions/factors" because anthropology is a discipline.
3. In the introduction lines 40-43 are redundant with the next paragraph ("in recent years"). Also, its main idea could be developed more by the authors, because forest fires have been modeling the landscapes, lives, and infrastructures since humans inhabited the planet, therefore, it is not an issue of "recent years".
4. In the introduction lines 45-46 are contradicting. If "natural processes have historically caused fires in forests" how could humans accelerate it? Also, what is a "firing process" that was accelerated?
5. In the study area lines 114-121 it could be explained more why "this province is known to be one of the most wildfire-prone regions in northern Iran" besides the reference to Adab et al. (2015). Also, the article could be benefited from a more detailed visual description of the topography (e.g. a topographic profile), which is a special feature remarked in the title "A mountainous case study area".
6. In the discussion section, the authors deliver more accurate information about the selected methods which improved the overall quality of the article and its scientific robustness. However, I missed a comparison with other studies doing the same methodologies in other mountain regions from Iran or other parts of the world with similar topography. Maybe the references named in lines 390-395 could be useful for that purpose if they are presented in a comparative table.
Technical corrections: Please be aware of the capital letter in the acronyms along the abstract (e.g., forest fire susceptibility map (FSM)). Also, the legend of figure 1 has a typo (boundaries).
Citation: https://doi.org/10.5194/egusphere-2022-1294-CC1 -
AC1: 'Reply on CC1', Parham Pahlavani, 30 Mar 2023
reply
Dear Reviewer,
We very much appreciate your positive statements regarding our manuscript. You indeed kindly spent significant time on this manuscript and we are grateful for the detailed suggestions. You brought up interesting aspects and we believe that these suggestions and our respective reactions to them will improve the quality of the paper. We did our best to improve the scientific quality of the manuscript significantly. Based on your constructive comments on the initial version and our comprehensive revisions, we are confident that you will find this version now worthwhile to get published. All the revisions are specified with the Blue color in the text.
1) The subtitle is a bit crowded: "A mountainous case study area".To comply with the reviewer’s comment, we changed the subtitle to " A study case in Iran".
2) In the abstract line 15 and section 3.2 the concept "anthropology/anthropological" should be replaced by "human actions/factors" or "anthropic actions/factors" because anthropology is a discipline.This comment was taken into consideration. In this regard, anthropology factors have been changed to human factors.
3) In the introduction lines 40-43 are redundant with the next paragraph ("in recent years"). Also, its main idea could be developed more by the authors, because forest fires have been modeling the landscapes, lives, and infrastructures since humans inhabited the planet, therefore, it is not an issue of "recent years".
To comply with the reviewer’s comment, the paragraph was corrected and its "recent years" was deleted in the revised version of the manuscript.
4) In the introduction lines 45-46 are contradicting. If "natural processes have historically caused fires in forests" how could humans accelerate it? Also, what is a "firing process" that was accelerated?To comply with the reviewer’s comment, the sentence was rewritten as follows:
“Fires in forests have accelerated directly due to the increased human-environmental interactions by igniting and suppressing fires, and indirectly by changing the vegetation structure and composition, as well as destroying the landscapes (Rogers et al., 2020).”
5) In the study area lines 114-121 it could be explained more why "this province is known to be one of the most wildfire-prone regions in northern Iran" besides the reference to Adab et al. (2015). Also, the article could be benefited from a more detailed visual description of the topography (e.g. a topographic profile), which is a special feature remarked in the title "A mountainous case study area".To point out the prevalence of fire in this area, the following sentence was added: “This study experienced several harsh wildfires, which have impacted more than twenty settlements and villages (Gholamnia et al., 2020).”
6) In the discussion section, the authors deliver more accurate information about the selected methods which improved the overall quality of the article and its scientific robustness. However, I missed a comparison with other studies doing the same methodologies in other mountain regions from Iran or other parts of the world with similar topography. Maybe the references named in lines 390-395 could be useful for that purpose if they are presented in a comparative table.If we pay attention to the discussion, we will find that it is explained in the full discussion section that several researchers have investigated factors, but the factor of distance to power lines is also known as an effective factor in this research. Also, the research conducted by Ghorbanzadeh in this study area has been mentioned and compared with the results of the research.
Finally, we appreciate your support and your constructive suggestions. We strongly believe that we have addressed the issues concerning the logical flow of the manuscript and have improved the scientific aspect significantly. We hope that you will find this revised version worthwhile to be published.
Thank you very much again.
The authors
Citation: https://doi.org/10.5194/egusphere-2022-1294-AC1 -
CC2: 'Reply on CC1', Parham Pahlavani, 30 Mar 2023
reply
Dear Reviewer,
We very much appreciate your positive statements regarding our manuscript. You indeed kindly spent significant time on this manuscript and we are grateful for the detailed suggestions. You brought up interesting aspects and we believe that these suggestions and our respective reactions to them will improve the quality of the paper. We did our best to improve the scientific quality of the manuscript significantly. Based on your constructive comments on the initial version and our comprehensive revisions, we are confident that you will find this version now worthwhile to get published. All the revisions are specified with the Blue color in the text.
1) The subtitle is a bit crowded: "A mountainous case study area".To comply with the reviewer’s comment, we changed the subtitle to " A study case in Iran".
2) In the abstract line 15 and section 3.2 the concept "anthropology/anthropological" should be replaced by "human actions/factors" or "anthropic actions/factors" because anthropology is a discipline.This comment was taken into consideration. In this regard, anthropology factors have been changed to human factors.
3) In the introduction lines 40-43 are redundant with the next paragraph ("in recent years"). Also, its main idea could be developed more by the authors, because forest fires have been modeling the landscapes, lives, and infrastructures since humans inhabited the planet, therefore, it is not an issue of "recent years".
To comply with the reviewer’s comment, the paragraph was corrected and its "recent years" was deleted in the revised version of the manuscript.
4) In the introduction lines 45-46 are contradicting. If "natural processes have historically caused fires in forests" how could humans accelerate it? Also, what is a "firing process" that was accelerated?To comply with the reviewer’s comment, the sentence was rewritten as follows:
“Fires in forests have accelerated directly due to the increased human-environmental interactions by igniting and suppressing fires, and indirectly by changing the vegetation structure and composition, as well as destroying the landscapes (Rogers et al., 2020).”
5) In the study area lines 114-121 it could be explained more why "this province is known to be one of the most wildfire-prone regions in northern Iran" besides the reference to Adab et al. (2015). Also, the article could be benefited from a more detailed visual description of the topography (e.g. a topographic profile), which is a special feature remarked in the title "A mountainous case study area".To point out the prevalence of fire in this area, the following sentence was added: “This study experienced several harsh wildfires, which have impacted more than twenty settlements and villages (Gholamnia et al., 2020).”
6) In the discussion section, the authors deliver more accurate information about the selected methods which improved the overall quality of the article and its scientific robustness. However, I missed a comparison with other studies doing the same methodologies in other mountain regions from Iran or other parts of the world with similar topography. Maybe the references named in lines 390-395 could be useful for that purpose if they are presented in a comparative table.If we pay attention to the discussion, we will find that it is explained in the full discussion section that several researchers have investigated factors, but the factor of distance to power lines is also known as an effective factor in this research. Also, the research conducted by Ghorbanzadeh in this study area has been mentioned and compared with the results of the research.
Finally, we appreciate your support and your constructive suggestions. We strongly believe that we have addressed the issues concerning the logical flow of the manuscript and have improved the scientific aspect significantly. We hope that you will find this revised version worthwhile to be published.
Thank you very much again.
The authors
Citation: https://doi.org/10.5194/egusphere-2022-1294-CC2 -
CC6: 'Reply on CC2', Parham Pahlavani, 30 Mar 2023
reply
The comment was uploaded in the form of a supplement: https://egusphere.copernicus.org/preprints/2023/egusphere-2022-1294/egusphere-2022-1294-CC6-supplement.pdf
-
CC6: 'Reply on CC2', Parham Pahlavani, 30 Mar 2023
reply
-
AC1: 'Reply on CC1', Parham Pahlavani, 30 Mar 2023
reply
-
RC1: 'Comment on egusphere-2022-1294', Anonymous Referee #1, 14 Mar 2023
reply
General overview of paper – there is some good work in here, but it reads too much like an thesis rather than a paper, thus inhibiting the communication. For instance – a lot of section 4 could/should be in the introduction, while the current introduction is verbose and not concise enough.
Additioanlly, in fire science slope, for instance, has been identified an important factor in wildfire modelling – however, your study area does not give details of slope or any of the 17 factors you’ve said are important. My inference therefore from your study and figure 7 is that the area is a) not overly sloped – which would be surprising for a mountainous region - or b) the fire areas and non-fire areas are equally sloped.
This then comes to the crux of the paper and its title – you are not modelling wildfire, you are modelling relative wildfire susceptibility in a certain area.
Lines 30-32 – provide reference to support claim.
Lines 33-34 – provide reference or specify that in this paper this is what you are calling a forest fire.
Line 35 – highly combustible trees compared to what?
Paragraph from line 33-42 – tell me why you are telling me this – why is this important.
Paragraph 43-53 – end sentences about life safety is new to para and out of generalised context of fires in Iran – this needs to be changed into two paras, with the latter being about human susceptibility.
Paragraph 54-64 – none of this is wrong, but it is not noteworthy – this is in my view – common knowledge.
Paragraph 64 – 79 – lots of repetition at the start, lots of “authors X did Y” and not enough depth to the review – a generic comment at the end about what is lacking without showing the reader that this is true.
Paragraph 80-104 – too long- consider splitting. As before lots of X did Y, not enough “however,…” there is some, but more needed.
Para 105 – 112 – need to explain why RFE is better than the other methods. Why have you chose this method. You then bring in terms such as adaboost, gradient boosting, random forests, and extra trees without any prior referencing in the text.
Resolution of Figure 1 could be improved. Please show an image from the timeframe being explored e.g., an image of the forest between 2012-2017
Can you explain the difference between a fire point and the polygons that you are using, and how they are used/interact with one another. Were there more than 352 fire points found and only 352 used or are you using all of the points that were found – I am confused as to why you have two different metrics to measure where a fire has been. How big is your fire point – what is the spatial resolution of your point?
How do you know the 17 factors are important – are there important factors that are not included? How has this list been determined or is it from other sources, in which case please reference these.
RFE – 4.1
Who is doing all this selecting/removing of features? Are these features independent of one another, or are they interdependent
Line 180 – cross-validation against what?
Again this comes back to the question about the factors/features of importance – knowing whether they are complete, and whether they are transferable between different geographical locations. This is not clear in the paper.
Line 194 & 207 – majority of votes – who’s voting?
195-201 – I do not see the importance of this part
205 What is a bootstrap sample – reference?
Please make the y axis in figures 3,4,5 and 6 the same so that the variability in each of the modelling approaches is abundantly clear. Do they need to be separate figures, it could be done on a single plot. What is the shading showing on these figures and why is this not discussed?
How are the models scored? What is the range of possible scores? is 0.87 considered acceptable?
Can the factors in table 2 and figure 7 please be in the same order
These are showing the same data – do these need to be duplicated?
Slope is an important factor in wildfire modelling – however, your study area does not give details of slope or any of the 17 factors you’ve said are important. My inference therefore from your study and figre 7 is that the area is not overly sloped – which would be surprising for a mountainous region - or the fire areas and non-fire areas are equally sloped.
This then comes to the crux of the paper and its title – you are not modelling wildfire, you are modelling relative wildfire susceptibility in a certain area.
Section 5.3 explains a statistical analysis of the different methods – it would be good to show what you mean by true and false positives. Tables 3-5 AUC are not described in the equations and not explained what it is – 6 equations 7 columns in the tables. There is no discussion over the merits of each method – so it just feels like noise.
The fire susceptibility maps – do these not just show that where there have been previous fires – these areas are susceptible to fires? There is no discussion of these images in any detail.
The discussion feels like an executive summary of the paper and the conclusions are lacking in conclusion.
Citation: https://doi.org/10.5194/egusphere-2022-1294-RC1 -
AC2: 'Reply on RC1', Parham Pahlavani, 30 Mar 2023
reply
Dear Reviewer,
We very much appreciate your positive statements regarding our manuscript. You indeed kindly spent significant time on this manuscript and we are grateful for the detailed suggestions. You brought up interesting aspects and we believe that these suggestions and our respective reactions to them will improve the quality of the paper. We did our best to improve the scientific quality of the manuscript significantly. Based on your constructive comments on the initial version and our comprehensive revisions, we are confident that you will find this version now worthwhile to get published. All the revisions are specified with the Blue color in the text.
1) A general overview of the paper – there is some good work in here, but it reads too much like a thesis rather than a paper, thus inhibiting communication. For instance – a lot of section 4 could/should be in the introduction, while the current introduction is verbose and not concise enough.According to other comments, modifications have been made in the introduction, which also covers this comment.
2) Additionally, in fire science slope, for instance, has been identified as an important factor in wildfire modeling – however, your study area does not give details of slope or any of the 17 factors you’ve said are important. My inference therefore from your study and figure 7 is that the area is a) not overly sloped – which would be surprising for a mountainous region - or b) the fire areas and non-fire areas are equally slopedWe added this paragraph: "In fire science, the slope factor has been identified as an important factor in fire modeling, but in the case of our study, there are many non-fire points in steep areas, so the models do not recognize this factor as an important factor in the fire forest. However, the direction of slope and height have been identified as important factors."
3) This then comes to the crux of the paper and its title – you are not modeling wildfire, you are modeling relative wildfire susceptibility in a certain area.The title of the article was changed to " How the recursive feature elimination affects the SVM and RF for relative wildfire susceptibility in a certain area? A study case in Iran ".
4) Lines 30-32 – provide reference to support the claim.
Add reference.
5) Lines 33-34 – provide a reference or specify that in this paper this is what you are calling a forest fire.Add reference.
6) Line 35 – highly combustible trees compared to what?Change to sentence to " Fires in forests lead to a lot of environmental destruction due to the presence of combustible vegetation "
7) Paragraph from lines 33-42 – tell me why you are telling me this – why is this important.
We added this sentence " The main goal of modeling forest fires is to reduce the negative effects of fires on humans and the environment as much as possible (Hosseini and Lim, 2022). Also, by determining the areas with the possibility of fire, it leads to better management of natural hazards (Tehrany et al., 2021)."
8) Paragraph 43-53 – end sentences about life safety is new to para and out of the generalized context of fires in Iran – this needs to be changed into two paras, with the latter being about human susceptibility.
Paragraphs 43 to 53 were changed into two paragraphs.
9) Paragraphs 54-64 – none of this is wrong, but it is not noteworthy – this is in my view – common knowledge.
The desired paragraphs were deleted.
10) Paragraphs 64 – 79 – lots of repetition at the start, lots of “authors X did Y” and not enough depth to the review – a generic comment at the end about what is lacking without showing the reader that this is true.
We added these sentences "in their study, the random forests (RFs) model was utilized to link historical fire events to a set of wildfire causative factors to measure the importance of each factor on fire ignition. then employed support vector machines (SVMs), to produce an accurate estimate of wildfire probability across the study area"
We added this sentence " this study utilizes Genetic Algorithms (GA) to obtain the optimal combination of forest fire-related variables and apply data mining methods for constructing a forest fire susceptibility map"
11) Paragraphs 80-104 – too long- consider splitting. As before lots of X did Y, not enough “however,…” there is some, but more needed.
Paragraphs 80-104 were divided into several paragraphs.
12) Para 105 – 112 – need to explain why RFE is better than the other methods. Why have you chosen this method? You then bring in terms such as adaboost, gradient boosting, random forests, and extra trees without any prior referencing in the text.
We added this paragraph " As a greedy way of finding a nested subset of features, RFE was argued to be much more robust to data overfitting than wrapper methods (Zeng et al., 2009). RFE tends to remove redundant and weak features and retains independent features. RFE seeks to improve generalization performance by removing the least important features whose deletion will have the least effect, on training errors (Escanilla et al., 2018)".
We added this paragraph ". Feature selection is accomplished by recursive feature elimination (RFE), As a result of its simplicity and effectiveness, RFE is a popular feature selection algorithm because it can identify the feature in a training dataset that is most relevant to the prediction of the variable."
13) Resolution of Figure 1 could be improved. Please show an image from the timeframe being explored e.g., an image of the forest between 2012-2017.
Image No. 1 was changed and considering that the fire points are related to the reported fire polygons and each reported fire polygon has a date since the fire occurred, a date label was added on the map for the fire points.
14) Can you explain the difference between a fire point and the polygons that you are using, and how they are used/interact with one another? Were there more than 352 fire points found and only 352 used or are you using all of the points that were found – I am confused as to why you have two different metrics to measure where a fire has been. How big is your fire point – what is the spatial resolution of your point?
We added this paragraph: "the fire polygons provided to us by the state wildlife organization of Amol County (SWOAC), were matched and validated with the MODIS fire data, and 352 fire points were found. Although there could be more than 352 fire points, it is not certain about them, but at these 352 fire points, fires have occurred with absolute certainty"
15) How do you know the 17 factors are important – are there important factors that are not included? How has this list been determined or is it from other sources, in which case please reference these.
We added this paragraph: "These factors have been extracted based on the studies that happened in the previous research, especially the studies that happened in this area. By reviewing articles (Ghorbanzadeh et al., 2019; Gigović et al., 2019; Zhang et al., 2019; Eskandari et al., 2021; Gholamnia et al., 2020; Jaafari and Pourghasemi, 2019), these 17 available factors were extracted."
16) Who is doing all this selecting/removing of features? Are these features independent of one another, or are they interdependent
As explained in the article, in this method, by building a model on the entire set of problem variables, it calculates their importance for each variable. In the next step, the least important variable is removed, the model is rebuilt on the other remaining variables, and the importance of each variable is recalculated.
17) Again, this comes back to the question about the factors/features of importance – knowing whether they are complete and whether they are transferable between different geographical locations. This is not clear in the paper.
In conclusion, we added this sentence " It is possible that the factors that are known as important factors in this study case, in another geographical environment, other factors are identified as important factors by this method, on the other hand, these factors that are known in this study case The factors that have been used were the ones that could be accessed, so it can be argued that there are other factors as well, but because they are not accessible, they have not been investigated."
18) Line 194 & 207 – the majority of votes – who’s voting?
where the predictions of individual trees are treated as votes and the prediction of the random forest is determined by the majority of votes.
19) 195-201 – I do not see the importance of this part
Due to the fact that the RFE method is described in these sentences, their presence is mandatory.
20) 205 What is a bootstrap sample – reference?
We added this sentence "The bootstrap sampling method is a resampling method that uses random sampling and bootstrap sample is a smaller sample that is “bootstrapped” from a larger sample (Efron and Tibshirani, 1993)".
21) Please make the y-axis in figures 3,4,5 and 6 the same so that the variability in each of the modeling approaches is abundantly clear. Do they need to be separate figures, it could be done on a single plot. What is the shading showing on these figures and why is this not discussed?
All four graphs are displayed in the same figure to enable comparison.
22) How are the models scored? What is the range of possible scores? is 0.87 considered acceptable?
The score obtained by the model here is the auc criterion. The auc range is between 0 and 1. The AUC values are interpreted as reflecting the following model accuracies: 0.6–0.7 poor, 0.6–0.7 medium, 0.7–0.8 good, 0.8–0.9 very good, and 0.9–1 excellent (Pourghasemi et al., 2017). According to Hong's article (Hong et al., 2018), which considered the value of 0.7 for the AUC criterion as a good performance of the model, also in the article of Jafari and Pourghasemi (Pourghasemi et al., 2020), the AUC criterion equal to 0.8 is considered a very good performance for the model.
23) Can the factors in table 2 and figure 7 please be in the same order
Figure 7, its order was arranged according to table 2.
24) These are showing the same data – do these need to be duplicated?
Due to the large number of features in Table 2, for easy identification of the votes obtained for each feature, Figure 7 is displayed as a graph of the votes obtained.
25) Section 5.3 explains a statistical analysis of the different methods – it would be good to show what you mean by true and false positives. Tables 3-5 AUC is not described in the equations and not explained what it is – 6 equations 7 columns in the tables. There is no discussion over the merits of each method – so it just feels like noise.
Formula AUC was added along with formulas Specificity and Sensitivity.
We added this paragraph: " The AUC criterion, which is one of the most important evaluation criteria for models, indicates that the proportion of fire and non-fire points which are correctly classified. The Recall measure shows how many positive examples in the sample are predicted correctly. The fraction of relevant instances in the retrieved instances. The sensitivity criterion also indicates the percentage of fire points that are correctly classified. On the other hand, the specificity criterion indicates the percentage of non-fire points that are correctly classified."
26) The fire susceptibility maps – do these not just show that where there have been previous fires – these areas are susceptible to fires? There is no discussion of these images in any detail.
In section 5.5 we added this paragraph: " A forest fire susceptibility map depicts areas likely to have forest fire in the future by correlating some of the principal factors that contribute to forest fire with the past distribution. The forest fire susceptibility maps represent a measure of the probability of the occurrence of wildfires for a region based on considered conditioning factors. The natural breaks classification method (available in Arc map 10.8) was used to classify the resulting spatial prediction of wildfire susceptibility maps. This classification method is the most common method for categorizing prediction maps for interpreting values close to each class boundary (e.g., values between “High” and “Very high” susceptibility predictions). The model generates a number between 0 and 1 for each pixel according to its feature vector. Using a reclassification tool in the Spatial Analyst Tools ArcGIS 10.8 software, each final map cell is classified into five classes (very low, low, moderate, high, and very high) representing the forest fire hazard index, with the natural breaks method, all outcomes are divided into five classes."
27) The discussion feels like an executive summary of the paper and the conclusions are lacking in conclusion.
According to other comments, modifications have been made in the discussion and conclusions, which also cover this comment.
These two paragraphs were added in the discussion:
" Pourghasemi et al. (Pourghasemi et al., 2020) have identified the factors of distance from rivers and residential areas, TWI, rainfall, aspect, and temperature as important factors in the Boruta algorithm and these factors have also been identified as important factors in our method. But the factors of use and slope, which are not known as important factors in the scope of our study, are known as influential factors in the scope of their study, and this indicates that in order to make a correct comparison between the methods of feature selection and To determine their performance, they should be tested in different geographical environments."
" In fire science, the slope factor has been identified as an important factor in fire modeling (Hong et al., 2018; Pourghasemi et al., 2020) , But there are also articles that do not consider the slope factor due to the geography of the region (Satir et al., 2016; Kim et al., 2019; Cao et al., 2017). About our study area, there are many non-fire points in steep areas, so the models do not recognize this factor as an important factor in the fire forest . However, the direction of slope and height have been identified as important factors. In the table below, the distribution of fire and non-fire points in the study area can be seen according to the slope classification. As can be inferred from the table, the distribution of fire and non-fire points in steep areas are similar, so the factor could not be recognized as an important fire factor in this study area."
The Table 8 was added.
And this paragraph was added in the conclusion:
"It is possible that the factors that are known as important factors in this study case, in another geographical environment, other factors are identified as important factors by this method, on the other hand, these factors that are known in this study case The factors that have been used were the ones that could be accessed, so it can be argued that there are other factors as well, but because they are not accessible, they have not been investigated"
Finally, we appreciate your support and your constructive suggestions. We strongly believe that we have addressed the issues concerning the logical flow of the manuscript and have improved the scientific aspect significantly. We hope that you will find this revised version worthwhile to be published.
Thank you very much again.
The authors
Citation: https://doi.org/10.5194/egusphere-2022-1294-AC2 -
CC3: 'Reply on RC1', Parham Pahlavani, 30 Mar 2023
reply
Dear Reviewer,
We very much appreciate your positive statements regarding our manuscript. You indeed kindly spent significant time on this manuscript and we are grateful for the detailed suggestions. You brought up interesting aspects and we believe that these suggestions and our respective reactions to them will improve the quality of the paper. We did our best to improve the scientific quality of the manuscript significantly. Based on your constructive comments on the initial version and our comprehensive revisions, we are confident that you will find this version now worthwhile to get published. All the revisions are specified with the Blue color in the text.
1) A general overview of the paper – there is some good work in here, but it reads too much like a thesis rather than a paper, thus inhibiting communication. For instance – a lot of section 4 could/should be in the introduction, while the current introduction is verbose and not concise enough.According to other comments, modifications have been made in the introduction, which also covers this comment.
2) Additionally, in fire science slope, for instance, has been identified as an important factor in wildfire modeling – however, your study area does not give details of slope or any of the 17 factors you’ve said are important. My inference therefore from your study and figure 7 is that the area is a) not overly sloped – which would be surprising for a mountainous region - or b) the fire areas and non-fire areas are equally slopedWe added this paragraph: "In fire science, the slope factor has been identified as an important factor in fire modeling, but in the case of our study, there are many non-fire points in steep areas, so the models do not recognize this factor as an important factor in the fire forest. However, the direction of slope and height have been identified as important factors."
3) This then comes to the crux of the paper and its title – you are not modeling wildfire, you are modeling relative wildfire susceptibility in a certain area.The title of the article was changed to " How the recursive feature elimination affects the SVM and RF for relative wildfire susceptibility in a certain area? A study case in Iran ".
4) Lines 30-32 – provide reference to support the claim.
Add reference.
5) Lines 33-34 – provide a reference or specify that in this paper this is what you are calling a forest fire.Add reference.
6) Line 35 – highly combustible trees compared to what?Change to sentence to " Fires in forests lead to a lot of environmental destruction due to the presence of combustible vegetation "
7) Paragraph from lines 33-42 – tell me why you are telling me this – why is this important.
We added this sentence " The main goal of modeling forest fires is to reduce the negative effects of fires on humans and the environment as much as possible (Hosseini and Lim, 2022). Also, by determining the areas with the possibility of fire, it leads to better management of natural hazards (Tehrany et al., 2021)."
8) Paragraph 43-53 – end sentences about life safety is new to para and out of the generalized context of fires in Iran – this needs to be changed into two paras, with the latter being about human susceptibility.
Paragraphs 43 to 53 were changed into two paragraphs.
9) Paragraphs 54-64 – none of this is wrong, but it is not noteworthy – this is in my view – common knowledge.
The desired paragraphs were deleted.
10) Paragraphs 64 – 79 – lots of repetition at the start, lots of “authors X did Y” and not enough depth to the review – a generic comment at the end about what is lacking without showing the reader that this is true.
We added these sentences "in their study, the random forests (RFs) model was utilized to link historical fire events to a set of wildfire causative factors to measure the importance of each factor on fire ignition. then employed support vector machines (SVMs), to produce an accurate estimate of wildfire probability across the study area"
We added this sentence " this study utilizes Genetic Algorithms (GA) to obtain the optimal combination of forest fire-related variables and apply data mining methods for constructing a forest fire susceptibility map"
11) Paragraphs 80-104 – too long- consider splitting. As before lots of X did Y, not enough “however,…” there is some, but more needed.
Paragraphs 80-104 were divided into several paragraphs.
12) Para 105 – 112 – need to explain why RFE is better than the other methods. Why have you chosen this method? You then bring in terms such as adaboost, gradient boosting, random forests, and extra trees without any prior referencing in the text.
We added this paragraph " As a greedy way of finding nested subset of features, RFE was argued to be much more robust to data overfitting than wrapper methods (Zeng et al., 2009). RFE tends to remove redundant and weak features and retains independent features. RFE seeks to improve generalization performance by removing the least important features whose deletion will have the least effect, on training errors (Escanilla et al., 2018)".
We added this paragraph ". Feature selection is accomplished by recursive feature elimination (RFE), As a result of its simplicity and effectiveness, RFE is a popular feature selection algorithm because it can identify the feature in a training dataset that is most relevant to the prediction of the variable."
13) Resolution of Figure 1 could be improved. Please show an image from the timeframe being explored e.g., an image of the forest between 2012-2017.
Image No. 1 was changed and considering that the fire points are related to the reported fire polygons and each reported fire polygon has a date since the fire occurred, a date label was added on the map for the fire points.
14) Can you explain the difference between a fire point and the polygons that you are using, and how they are used/interact with one another? Were there more than 352 fire points found and only 352 used or are you using all of the points that were found – I am confused as to why you have two different metrics to measure where a fire has been. How big is your fire point – what is the spatial resolution of your point?
We added this paragraph: "the fire polygons provided to us by the state wildlife organization of Amol County (SWOAC), were matched and validated with the MODIS fire data, and 352 fire points were found. Although there could be more than 352 fire points, it is not certain about them, but at these 352 fire points, fires have occurred with absolute certainty"
15) How do you know the 17 factors are important – are there important factors that are not included? How has this list been determined or is it from other sources, in which case please reference these.
We added this paragraph: "These factors have been extracted based on the studies that happened in the previous research, especially the studies that happened in this area. By reviewing articles (Ghorbanzadeh et al., 2019; Gigović et al., 2019; Zhang et al., 2019; Eskandari et al., 2021; Gholamnia et al., 2020; Jaafari and Pourghasemi, 2019), these 17 available factors were extracted."
16) Who is doing all this selecting/removing of features? Are these features independent of one another, or are they interdependent
As explained in the article, in this method, by building a model on the entire set of problem variables, it calculates their importance for each variable. In the next step, the least important variable is removed, the model is rebuilt on the other remaining variables, and the importance of each variable is recalculated.
17) Again, this comes back to the question about the factors/features of importance – knowing whether they are complete and whether they are transferable between different geographical locations. This is not clear in the paper.
In conclusion, we added this sentence " It is possible that the factors that are known as important factors in this study case, in another geographical environment, other factors are identified as important factors by this method, on the other hand, these factors that are known in this study case The factors that have been used were the ones that could be accessed, so it can be argued that there are other factors as well, but because they are not accessible, they have not been investigated."
18) Line 194 & 207 – the majority of votes – who’s voting?
where the predictions of individual trees are treated as votes and the prediction of the random forest is determined by the majority of votes.
19) 195-201 – I do not see the importance of this part
Due to the fact that the RFE method is described in these sentences, their presence is mandatory.
20) 205 What is a bootstrap sample – reference?
We added this sentence "The bootstrap sampling method is a resampling method that uses random sampling and bootstrap sample is a smaller sample that is “bootstrapped” from a larger sample (Efron and Tibshirani, 1993)".
21) Please make the y-axis in figures 3,4,5 and 6 the same so that the variability in each of the modeling approaches is abundantly clear. Do they need to be separate figures, it could be done on a single plot. What is the shading showing on these figures and why is this not discussed?
All four graphs are displayed in the same figure to enable comparison.
22) How are the models scored? What is the range of possible scores? is 0.87 considered acceptable?
The score obtained by the model here is the auc criterion. The auc range is between 0 and 1. The AUC values are interpreted as reflecting the following model accuracies: 0.6–0.7 poor, 0.6–0.7 medium, 0.7–0.8 good, 0.8–0.9 very good, and 0.9–1 excellent (Pourghasemi et al., 2017). According to Hong's article (Hong et al., 2018), which considered the value of 0.7 for the AUC criterion as a good performance of the model, also in the article of Jafari and Pourghasemi (Pourghasemi et al., 2020), the AUC criterion equal to 0.8 is considered a very good performance for the model.
23) Can the factors in table 2 and figure 7 please be in the same order
Figure 7, its order was arranged according to table 2.
24) These are showing the same data – do these need to be duplicated?
Due to the large number of features in Table 2, for easy identification of the votes obtained for each feature, Figure 7 is displayed as a graph of the votes obtained.
25) Section 5.3 explains a statistical analysis of the different methods – it would be good to show what you mean by true and false positives. Tables 3-5 AUC is not described in the equations and not explained what it is – 6 equations 7 columns in the tables. There is no discussion over the merits of each method – so it just feels like noise.
Formula AUC was added along with formulas Specificity and Sensitivity.
We added this paragraph: " The AUC criterion, which is one of the most important evaluation criteria for models, indicates that the proportion of fire and non-fire points which are correctly classified. The Recall measure shows how many positive examples in the sample are predicted correctly. The fraction of relevant instances in the retrieved instances. The sensitivity criterion also indicates the percentage of fire points that are correctly classified. On the other hand, the specificity criterion indicates the percentage of non-fire points that are correctly classified."
26) The fire susceptibility maps – do these not just show that where there have been previous fires – these areas are susceptible to fires? There is no discussion of these images in any detail.
In section 5.5 we added this paragraph: " A forest fire susceptibility map depicts areas likely to have forest fire in the future by correlating some of the principal factors that contribute to forest fire with the past distribution. The forest fire susceptibility maps represent a measure of the probability of the occurrence of wildfires for a region based on considered conditioning factors. The natural breaks classification method (available in Arc map 10.8) was used to classify the resulting spatial prediction of wildfire susceptibility maps. This classification method is the most common method for categorizing prediction maps for interpreting values close to each class boundary (e.g., values between “High” and “Very high” susceptibility predictions). The model generates a number between 0 and 1 for each pixel according to its feature vector. Using a reclassification tool in the Spatial Analyst Tools ArcGIS 10.8 software, each final map cell is classified into five classes (very low, low, moderate, high, and very high) representing the forest fire hazard index, with the natural breaks method, all outcomes are divided into five classes."
27) The discussion feels like an executive summary of the paper and the conclusions are lacking in conclusion.
According to other comments, modifications have been made in the discussion and conclusions, which also cover this comment.
These two paragraphs were added in the discussion:
" Pourghasemi et al. (Pourghasemi et al., 2020) have identified the factors of distance from rivers and residential areas, TWI, rainfall, aspect, and temperature as important factors in the Boruta algorithm and these factors have also been identified as important factors in our method. But the factors of use and slope, which are not known as important factors in the scope of our study, are known as influential factors in the scope of their study, and this indicates that in order to make a correct comparison between the methods of feature selection and To determine their performance, they should be tested in different geographical environments."
" In fire science, the slope factor has been identified as an important factor in fire modeling (Hong et al., 2018; Pourghasemi et al., 2020) , But there are also articles that do not consider the slope factor due to the geography of the region (Satir et al., 2016; Kim et al., 2019; Cao et al., 2017). About our study area, there are many non-fire points in steep areas, so the models do not recognize this factor as an important factor in the fire forest . However, the direction of slope and height have been identified as important factors. In the table below, the distribution of fire and non-fire points in the study area can be seen according to the slope classification. As can be inferred from the table, the distribution of fire and non-fire points in steep areas are similar, so the factor could not be recognized as an important fire factor in this study area."
The Table 8 was added.
And this paragraph was added in the conclusion:
"It is possible that the factors that are known as important factors in this study case, in another geographical environment, other factors are identified as important factors by this method, on the other hand, these factors that are known in this study case The factors that have been used were the ones that could be accessed, so it can be argued that there are other factors as well, but because they are not accessible, they have not been investigated"
Finally, we appreciate your support and your constructive suggestions. We strongly believe that we have addressed the issues concerning the logical flow of the manuscript and have improved the scientific aspect significantly. We hope that you will find this revised version worthwhile to be published.
Thank you very much again.
The authors
Citation: https://doi.org/10.5194/egusphere-2022-1294-CC3 -
CC4: 'Reply on RC1', Parham Pahlavani, 30 Mar 2023
reply
Dear Reviewer,
We very much appreciate your positive statements regarding our manuscript. You indeed kindly spent significant time on this manuscript and we are grateful for the detailed suggestions. You brought up interesting aspects and we believe that these suggestions and our respective reactions to them will improve the quality of the paper. We did our best to improve the scientific quality of the manuscript significantly. Based on your constructive comments on the initial version and our comprehensive revisions, we are confident that you will find this version now worthwhile to get published. All the revisions are specified with the Blue color in the text.
1) A general overview of the paper – there is some good work in here, but it reads too much like a thesis rather than a paper, thus inhibiting communication. For instance – a lot of section 4 could/should be in the introduction, while the current introduction is verbose and not concise enough.According to other comments, modifications have been made in the introduction, which also covers this comment.
2) Additionally, in fire science slope, for instance, has been identified as an important factor in wildfire modeling – however, your study area does not give details of slope or any of the 17 factors you’ve said are important. My inference therefore from your study and figure 7 is that the area is a) not overly sloped – which would be surprising for a mountainous region - or b) the fire areas and non-fire areas are equally slopedWe added this paragraph: "In fire science, the slope factor has been identified as an important factor in fire modeling, but in the case of our study, there are many non-fire points in steep areas, so the models do not recognize this factor as an important factor in the fire forest. However, the direction of slope and height have been identified as important factors."
3) This then comes to the crux of the paper and its title – you are not modeling wildfire, you are modeling relative wildfire susceptibility in a certain area.The title of the article was changed to " How the recursive feature elimination affects the SVM and RF for relative wildfire susceptibility in a certain area? A study case in Iran ".
4) Lines 30-32 – provide reference to support the claim.
Add reference.
5) Lines 33-34 – provide a reference or specify that in this paper this is what you are calling a forest fire.Add reference.
6) Line 35 – highly combustible trees compared to what?Change to sentence to " Fires in forests lead to a lot of environmental destruction due to the presence of combustible vegetation "
7) Paragraph from lines 33-42 – tell me why you are telling me this – why is this important.
We added this sentence " The main goal of modeling forest fires is to reduce the negative effects of fires on humans and the environment as much as possible (Hosseini and Lim, 2022). Also, by determining the areas with the possibility of fire, it leads to better management of natural hazards (Tehrany et al., 2021)."
8) Paragraph 43-53 – end sentences about life safety is new to para and out of the generalized context of fires in Iran – this needs to be changed into two paras, with the latter being about human susceptibility.
Paragraphs 43 to 53 were changed into two paragraphs.
9) Paragraphs 54-64 – none of this is wrong, but it is not noteworthy – this is in my view – common knowledge.
The desired paragraphs were deleted.
10) Paragraphs 64 – 79 – lots of repetition at the start, lots of “authors X did Y” and not enough depth to the review – a generic comment at the end about what is lacking without showing the reader that this is true.
We added these sentences "in their study, the random forests (RFs) model was utilized to link historical fire events to a set of wildfire causative factors to measure the importance of each factor on fire ignition. then employed support vector machines (SVMs), to produce an accurate estimate of wildfire probability across the study area"
We added this sentence " this study utilizes Genetic Algorithms (GA) to obtain the optimal combination of forest fire-related variables and apply data mining methods for constructing a forest fire susceptibility map"
11) Paragraphs 80-104 – too long- consider splitting. As before lots of X did Y, not enough “however,…” there is some, but more needed.
Paragraphs 80-104 were divided into several paragraphs.
12) Para 105 – 112 – need to explain why RFE is better than the other methods. Why have you chosen this method? You then bring in terms such as adaboost, gradient boosting, random forests, and extra trees without any prior referencing in the text.
We added this paragraph " As a greedy way of finding nested subset of features, RFE was argued to be much more robust to data overfitting than wrapper methods (Zeng et al., 2009). RFE tends to remove redundant and weak features and retains independent features. RFE seeks to improve generalization performance by removing the least important features whose deletion will have the least effect, on training errors (Escanilla et al., 2018)".
We added this paragraph ". Feature selection is accomplished by recursive feature elimination (RFE), As a result of its simplicity and effectiveness, RFE is a popular feature selection algorithm because it can identify the feature in a training dataset that is most relevant to the prediction of the variable."
13) Resolution of Figure 1 could be improved. Please show an image from the timeframe being explored e.g., an image of the forest between 2012-2017.
Image No. 1 was changed and considering that the fire points are related to the reported fire polygons and each reported fire polygon has a date since the fire occurred, a date label was added on the map for the fire points.
14) Can you explain the difference between a fire point and the polygons that you are using, and how they are used/interact with one another? Were there more than 352 fire points found and only 352 used or are you using all of the points that were found – I am confused as to why you have two different metrics to measure where a fire has been. How big is your fire point – what is the spatial resolution of your point?
We added this paragraph: "the fire polygons provided to us by the state wildlife organization of Amol County (SWOAC), were matched and validated with the MODIS fire data, and 352 fire points were found. Although there could be more than 352 fire points, it is not certain about them, but at these 352 fire points, fires have occurred with absolute certainty"
15) How do you know the 17 factors are important – are there important factors that are not included? How has this list been determined or is it from other sources, in which case please reference these.
We added this paragraph: "These factors have been extracted based on the studies that happened in the previous research, especially the studies that happened in this area. By reviewing articles (Ghorbanzadeh et al., 2019; Gigović et al., 2019; Zhang et al., 2019; Eskandari et al., 2021; Gholamnia et al., 2020; Jaafari and Pourghasemi, 2019), these 17 available factors were extracted."
16) Who is doing all this selecting/removing of features? Are these features independent of one another, or are they interdependent
As explained in the article, in this method, by building a model on the entire set of problem variables, it calculates their importance for each variable. In the next step, the least important variable is removed, the model is rebuilt on the other remaining variables, and the importance of each variable is recalculated.
17) Again, this comes back to the question about the factors/features of importance – knowing whether they are complete and whether they are transferable between different geographical locations. This is not clear in the paper.
In conclusion, we added this sentence " It is possible that the factors that are known as important factors in this study case, in another geographical environment, other factors are identified as important factors by this method, on the other hand, these factors that are known in this study case The factors that have been used were the ones that could be accessed, so it can be argued that there are other factors as well, but because they are not accessible, they have not been investigated."
18) Line 194 & 207 – the majority of votes – who’s voting?
where the predictions of individual trees are treated as votes and the prediction of the random forest is determined by the majority of votes.
19) 195-201 – I do not see the importance of this part
Due to the fact that the RFE method is described in these sentences, their presence is mandatory.
20) 205 What is a bootstrap sample – reference?
We added this sentence "The bootstrap sampling method is a resampling method that uses random sampling and bootstrap sample is a smaller sample that is “bootstrapped” from a larger sample (Efron and Tibshirani, 1993)".
21) Please make the y-axis in figures 3,4,5 and 6 the same so that the variability in each of the modeling approaches is abundantly clear. Do they need to be separate figures, it could be done on a single plot. What is the shading showing on these figures and why is this not discussed?
All four graphs are displayed in the same figure to enable comparison.
22) How are the models scored? What is the range of possible scores? is 0.87 considered acceptable?
The score obtained by the model here is the auc criterion. The auc range is between 0 and 1. The AUC values are interpreted as reflecting the following model accuracies: 0.6–0.7 poor, 0.6–0.7 medium, 0.7–0.8 good, 0.8–0.9 very good, and 0.9–1 excellent (Pourghasemi et al., 2017). According to Hong's article (Hong et al., 2018), which considered the value of 0.7 for the AUC criterion as a good performance of the model, also in the article of Jafari and Pourghasemi (Pourghasemi et al., 2020), the AUC criterion equal to 0.8 is considered a very good performance for the model.
23) Can the factors in table 2 and figure 7 please be in the same order
Figure 7, its order was arranged according to table 2.
24) These are showing the same data – do these need to be duplicated?
Due to the large number of features in Table 2, for easy identification of the votes obtained for each feature, Figure 7 is displayed as a graph of the votes obtained.
25) Section 5.3 explains a statistical analysis of the different methods – it would be good to show what you mean by true and false positives. Tables 3-5 AUC is not described in the equations and not explained what it is – 6 equations 7 columns in the tables. There is no discussion over the merits of each method – so it just feels like noise.
Formula AUC was added along with formulas Specificity and Sensitivity.
We added this paragraph: " The AUC criterion, which is one of the most important evaluation criteria for models, indicates that the proportion of fire and non-fire points which are correctly classified. The Recall measure shows how many positive examples in the sample are predicted correctly. The fraction of relevant instances in the retrieved instances. The sensitivity criterion also indicates the percentage of fire points that are correctly classified. On the other hand, the specificity criterion indicates the percentage of non-fire points that are correctly classified."
26) The fire susceptibility maps – do these not just show that where there have been previous fires – these areas are susceptible to fires? There is no discussion of these images in any detail.
In section 5.5 we added this paragraph: " A forest fire susceptibility map depicts areas likely to have forest fire in the future by correlating some of the principal factors that contribute to forest fire with the past distribution. The forest fire susceptibility maps represent a measure of the probability of the occurrence of wildfires for a region based on considered conditioning factors. The natural breaks classification method (available in Arc map 10.8) was used to classify the resulting spatial prediction of wildfire susceptibility maps. This classification method is the most common method for categorizing prediction maps for interpreting values close to each class boundary (e.g., values between “High” and “Very high” susceptibility predictions). The model generates a number between 0 and 1 for each pixel according to its feature vector. Using a reclassification tool in the Spatial Analyst Tools ArcGIS 10.8 software, each final map cell is classified into five classes (very low, low, moderate, high, and very high) representing the forest fire hazard index, with the natural breaks method, all outcomes are divided into five classes."
27) The discussion feels like an executive summary of the paper and the conclusions are lacking in conclusion.
According to other comments, modifications have been made in the discussion and conclusions, which also cover this comment.
These two paragraphs were added in the discussion:
" Pourghasemi et al. (Pourghasemi et al., 2020) have identified the factors of distance from rivers and residential areas, TWI, rainfall, aspect, and temperature as important factors in the Boruta algorithm and these factors have also been identified as important factors in our method. But the factors of use and slope, which are not known as important factors in the scope of our study, are known as influential factors in the scope of their study, and this indicates that in order to make a correct comparison between the methods of feature selection and To determine their performance, they should be tested in different geographical environments."
" In fire science, the slope factor has been identified as an important factor in fire modeling (Hong et al., 2018; Pourghasemi et al., 2020) , But there are also articles that do not consider the slope factor due to the geography of the region (Satir et al., 2016; Kim et al., 2019; Cao et al., 2017). About our study area, there are many non-fire points in steep areas, so the models do not recognize this factor as an important factor in the fire forest . However, the direction of slope and height have been identified as important factors. In the table below, the distribution of fire and non-fire points in the study area can be seen according to the slope classification. As can be inferred from the table, the distribution of fire and non-fire points in steep areas are similar, so the factor could not be recognized as an important fire factor in this study area."
The Table 8 was added.
And this paragraph was added in the conclusion:
"It is possible that the factors that are known as important factors in this study case, in another geographical environment, other factors are identified as important factors by this method, on the other hand, these factors that are known in this study case The factors that have been used were the ones that could be accessed, so it can be argued that there are other factors as well, but because they are not accessible, they have not been investigated"
Finally, we appreciate your support and your constructive suggestions. We strongly believe that we have addressed the issues concerning the logical flow of the manuscript and have improved the scientific aspect significantly. We hope that you will find this revised version worthwhile to be published.
Thank you very much again.
The authors
Citation: https://doi.org/10.5194/egusphere-2022-1294-CC4 -
CC5: 'Reply on CC4', Parham Pahlavani, 30 Mar 2023
reply
The comment was uploaded in the form of a supplement: https://egusphere.copernicus.org/preprints/2023/egusphere-2022-1294/egusphere-2022-1294-CC5-supplement.pdf
-
CC5: 'Reply on CC4', Parham Pahlavani, 30 Mar 2023
reply
-
AC2: 'Reply on RC1', Parham Pahlavani, 30 Mar 2023
reply
-
CC7: 'Comment on egusphere-2022-1294', F. Hosseinali, 01 Jun 2023
reply
I encountered a question when I read your article: How did you assess the independence of input factors such as vegetation, height, etc?
Citation: https://doi.org/10.5194/egusphere-2022-1294-CC7
Ali Rezaei Barzani et al.
Ali Rezaei Barzani et al.
Viewed
| HTML | XML | Total | BibTeX | EndNote | |
|---|---|---|---|---|---|
| 276 | 95 | 25 | 396 | 3 | 4 |
- HTML: 276
- PDF: 95
- XML: 25
- Total: 396
- BibTeX: 3
- EndNote: 4
Viewed (geographical distribution)
| Country | # | Views | % |
|---|
| Total: | 0 |
| HTML: | 0 |
| PDF: | 0 |
| XML: | 0 |
- 1