the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
Multifidelity Monte Carlo Estimation for Efficient Uncertainty Quantification in ClimateRelated Modeling
Anthony Gruber
Max Gunzburger
Lili Ju
Rihui Lan
Zhu Wang
Abstract. Uncertainties in an output of interest that depends on the solution of a complex system (e.g., of partial differential equations with random inputs) are often, if not nearly ubiquitously, determined in practice using Monte Carlo (MC) estimation. While simple to implement, MC estimation fails to provide reliable information about statistical quantities (such as the expected value of the output of interest) in application settings such as climate modeling for which obtaining a single realization of the output of interest is a costly endeavor. Specifically, the dilemma encountered is that many samples of the output of interest have to be collected in order to obtain an MC estimator having sufficient accuracy; so many, in fact, that the available computational budget is not large enough to effect the number of samples needed. To circumvent this dilemma, we consider using multifidelity Monte Carlo (MFMC) estimation which leverages the use of less costly and less accurate surrogate models (such as coarser grids, reducedorder models, simplified physics, interpolants, etc.) to achieve, for the same computational budget, higher accuracy compared to that obtained by an MC estimator or, looking at it another way, an MFMC estimator obtains the same accuracy as the MC estimator at lower computational cost. The key to the efficacy of MFMC estimation is the fact that most of the required computational budget is loaded onto the less costly surrogate models, so that very few samples are taken of the more expensive model of interest. We first provide a more detailed discussion about the need to consider an alternate to MC estimation for uncertainty quantification. Subsequently, we present a review, in an abstract setting, of the MFMC approach along with its application to three climaterelated benchmark problems as a proofofconcept exercise.

Notice on discussion status
The requested preprint has a corresponding peerreviewed final revised paper. You are encouraged to refer to the final revised version.

Preprint
(3762 KB)

The requested preprint has a corresponding peerreviewed final revised paper. You are encouraged to refer to the final revised version.
 Preprint
(3762 KB)  BibTeX
 EndNote
 Final revised paper
Journal article(s) based on this preprint
Anthony Gruber et al.
Interactive discussion
Status: closed

RC1: 'Comment on egusphere2022797', Anonymous Referee #1, 09 Oct 2022
Review Report on egusphere2022797
"Multifidelity Monte Carlo Estimation for Efficient Uncertainty Quantification in ClimateRelated Modeling"
by Anthony Gruber, Max Gunzburger, Lili Ju, Rihui Lan, and Zhu Wang
In this paper, the authors extensively tested the multifidelity Monte Carlo (MFMC) method for accurately catching some quantities of interest in climaterelated modeling. Compared to the widely used Monte Carlo (MC) method, they demonstrated that MFMC has superior properties over MC on this subject via three benchmark problems in the ocean, atmosphere, and ice sheet modeling. Since uncertainties exist almost everywhere in climate modeling and prediction, I think MFMC would greatly impact this area, especially in reducing the number of costly largescale simulations in very fine resolutions. Hence, I would highly recommend the publication of this manuscript in GMD after the authors address the following minor issues.
 In line 67 on page 3, the statement is somehow confusing. The authors declare that they don't consider any possible alternate sampling schemes at the beginning. But their goal is to use a nontraditional MC sampling strategy. Doesn't the sampling strategy belong to the "alternate sampling schemes"?
 In Algorithm A, it mentions that F^{k} is removed from the list of surrogates if the second requirement is not satisfied. Will there be such a case all F^{k} will be removed from the estimation? Secondly, will it obtain less accuracy compared to the MC method with all F^{k }included?
 For Figures 3 and 6, it is better to point out that the blue shading is the standard deviation of the MC prediction and the melon for MFMC. Moreover, being included in the blue shading indicates a smaller standard deviation.
Citation: https://doi.org/10.5194/egusphere2022797RC1 
AC1: 'Reply on RC1', Anthony Gruber, 10 Oct 2022
Dear Anonymous Referee #1,Thank you for your helpful comments and for your interest in our work. Below are responses to your specific questions:1) By “alternate sampling strategies” we mean that a user might be using (or may want to use) some other sampling scheme besides traditional MC for the truth model, e.g., Latin hypercube sampling, quasiMC sampling, or sparse grid sampling, as doing so often results in lower costs for a given accuracy tolerance. Therefore, the choice of sampling scheme is determined by how we want to sample the truth model and its surrogates. While we use MC sampling in this work, other choices are possible and would produce analogues to the MFMC method, e.g., MFLH, MFQMC, or MFSG.
2) Note that all F^{k }will never be removed according to our criterion; in the most extreme case, at least the highfidelity model F^{1} will remain. On the other hand, if this occurs, we obtain the useful information that none of the surrogates F^{2}, …, F^{k} are effective at increasing estimation accuracy or reducing computational costs, i.e., we are stuck using MC on its own and should look for other, more informative, surrogates. To address the other question, there should never be a loss of accuracy relative to MC regardless of the model set. The worstcase scenario is if all the surrogates fail the second test, in which case we are left with traditional MC using F^{1}. Hence, there is no loss of accuracy in MFMC; its worstcase accuracy is simply equal to that of MC using the truth model. However, if at least one surrogate survives the second test, then the MFMC method guarantees that costs are reduced without compromising accuracy.
3) We agree that it is a good idea to mention this explicitly. Any future versions of the manuscript will include this change.Thanks again, and please let us know if you have other questions.Best,
Anthony, Max, Lili, Rihui, and ZhuCitation: https://doi.org/10.5194/egusphere2022797AC1

RC2: 'Comment on egusphere2022797', Anonymous Referee #2, 31 Oct 2022
Review for Multifidelity Monte Carlo Estimation for Efficient Uncertainty Quantification in ClimateRelated Modeling
The authors describe a "multifidelity" Monte Carlo method for expected value estimation that is shown to improve convergence/efficiency compared to traditional Monte Carlo methods. This multifidelity approach is different to existing techniques in that expected values are computed using simulations run at differing resolutions and/or incorporating various levels of dynamical approximation. The test cases presented here employ hierarchies of progressively coarsened meshes to provide lowcost estimators, as well as an interpolationbased approach. The authors present results for two shallowwater benchmarks and one icesheet testcase, and demonstrate significant speedups compared to the traditional MC scheme for various maximumvalue estimation problems. The speedups demonstrated for the MFMC method are very attractive, especially in the context of climatemodel estimators which are often extremely expensive to obtain. There are a number of questions I have relating MFMC performance to problem nonlinearity though, in addition to various minor corrections.
Comments: The three testcases analysed appear to involve estimators evaluated after relatively short integration periods, and, rather than fully nonlinear flows/dynamics, the response of the system may be relatively linear over these short horizons as a result  is this expected to influence the effectiveness of the MFMC results presented? For example, TC5 from Williamson does not become strongly nonlinear until approx. 20days, with 50days being the typical analysis window at which turbulence is fully developed. The SOMA case described in Wolfram (2015) is typically spunup over several years, and then analysed over 30day windows. In this work, it appears the TC5 case is analysed after 10 days, and the barotropicgyreversionofSOMA after 3days (restarted from a 15day spinup). Are the MFMC results robust when the duration/nonlinearity of the testcases is increased?
 Is it possible to estimate the relative "multifidelity" contributions to the accuracy of the MFMC estimator? For example, is the overall accuracy governed more by the small number of high resolution runs, the large number of low resolution runs, or something inbetween? Considering the more linear (or at least nonturbulent) nature of solutions studied, how would a conventional MC estimator compare if run only using lowerresolution simulations? In other words, is the good performance of the MFMC method due to the solution being wellresolved even on the coarser meshes, or is the multiresolution hierarchy effective in estimating behaviour resolvable only at highresolution? If it is the former, I wonder whether the problems studied are sufficiently nonlinear at the gridscale. If it is the latter, this may be a nice result to highlight further.
 I believe the gradient terms in the shallowwater system (12) should be grad(1/2*u^2) + g*grad(h + h_b) rather than the grad(rho) included currently. Here p = rho_0*g*h is used to simplify the linear 1/rho_0 * grad(p) shallowwater pressure gradient, consistent with e.g. Ringler et al (2010).
 The SWE runtimes noted in 3.1.1 and 3.2.1 appear to be quite slow  requiring 100's of seconds to advance a single timestep using relatively small O(<= 100,000) cell meshes? Are these runtimes for the full multiday simulations instead, or for all ensemble members perhaps?
 While the MFMC methods presented here are clearly different in that they leverage varying resolution simulations, is it fair to compare against only the "historical" MC method, which is known to be uncompetitive in terms of efficiency? Significant work on alternative MC methods has been conducted by various authors in which a variety of accelerated techniques have been proposed. Are the large gains reported for MFMC expected to be replicated compared to e.g. MCMC (Markov Chain Monte Carlo) approaches more frequently used in climate model estimation?
Minor comments: The SOMA test case (Simulating Ocean Mesoscale Activity) typically refers to simulations using the multilayer primitive equations, in which mesoscale eddies form due to 3d interactions between the momentum, density and forcing tendencies. In this shallowwater configuration with rho = const., it appears to be a winddriven barotropic gyre that's studied instead, which is typically less turbulent, as per the smooth flow features in fig. 2. If so, it's suggested to label this test case as a winddriven gyre.
 Wallis (2012) reference appears to be missing.
 ln 76: Is saying "no guesswork involved" too strong a statement? The systematic nature of the MFMC approach is attractive, but is it *the* provably optimal sampling strategy, or more of an effective heuristic?
 ln 72: ...also uses cheaper to obtain...
 fig. 3 labelling: leftright vs topbottom.
 ln 308: Is this an expression for the free surface height or the layer thickness  h appears to be thickness in the shallowwater system (12).
 The Gruber (2022) paper referenced here appears to be an arXiv preprint, that inturn references this GMD submission??
Citation: https://doi.org/10.5194/egusphere2022797RC2 
AC2: 'Response to Anonymous Referee #2', Anthony Gruber, 11 Nov 2022
Dear Anonymous Referee #2,
Thank you for your helpful comments and for your interest in our work. Below are responses to your specific comments, numbered in order of appearance. We are also attaching a colorcoded document with the same information in case the mapping between your comments and our responses is not clear.
1) Based on our experience with prior work on MCbased estimation, we do not expect
the linearity (or lack thereof) of the model solution to affect the performance of the
MFMC estimation procedure.2) A goal of MFMC estimation is to achieve (using very few samples of the highfidelity
model) the same accuracy as obtained by an MC estimator (that exclusively uses many
samples of that model). Certainly, at least one highfidelity model evaluation is
necessary to eliminate bias in the MFMC estimator. Moreover, we find practically that
the highfidelity samples used to “steer” this estimator in an accurate direction, while
the lowfidelity samples are used to shrink its variance around the true solution. Note
that the dynamics of the example systems are not particularly well resolved by the low
fidelity models relative to the highfidelity ones; particularly in the case of the
barotropic gyre (SOMA) case, there is a noticeable visual difference between the 32km
solution and the 8km solution.3) This has now been corrected. Thank you for your attention.
4) These were the wallclock times observed when the relevant system was implemented in MATLAB and run on a 2015 MacBook laptop. Therefore, neither the implementation nor the hardware was optimized for computational efficiency.
5) A primary benefit of MFMC over other modern estimation methods such as MCMC is its ability to leverage lowfidelity information to effect costsavings without sacrificing estimator accuracy. In our experience with MFMC in other settings, this benefit translates to much larger cost savings for a given accuracy tolerance when compared to MCMC as well as other MCrelated sampling schemes (e.g., variance reduction MC, importance sampling).
Minor comments:
1) We agree with this reasoning and have changed the name globally throughout the manuscript.
2) This has been fixed, thank you for your attention.
3) It can be shown that the MFMC method presented here is the provably optimal solution (up to rounding) to a particular constrained optimization problem (see Gruber et al 2022, “A multifidelity Monte Carlo method for realistic computational budgets”, for a formal statement). Therefore, we do not think it is a stretch to say there is “no guesswork involved” in this context.
4, 5) These have been fixed, thank you for your attention.
6) This is an expression for the fluid thickness. We have clarified this globally throughout the manuscript.
7) This is true. The referenced preprint (to appear in J. Sci. Comput.) establishes the particular MFMC algorithm which has been applied to climaterelated examples in this paper. Therefore, we refer interested readers to that manuscript for a more detailed description of the MFMC method. Conversely, we write in that preprint that “Forthcoming work will investigate applications of the present MFMC method to complex systems governed by partial differential equations, particularly in the context of climate modeling.”, and provide an empty citation with title and relevant authors. The mentioned work has since become this GMD submission. Since this has caused confusion, we intend to remove the offending citation from the J. Sci. Comput. paper during the proofing stage.
Thanks again, and please let us know if you have other questions.Best,
Anthony, Max, Lili, Rihui, and Zhu

AC2: 'Response to Anonymous Referee #2', Anthony Gruber, 11 Nov 2022

AC2: 'Response to Anonymous Referee #2', Anthony Gruber, 11 Nov 2022
Dear Anonymous Referee #2,
Thank you for your helpful comments and for your interest in our work. Below are responses to your specific comments, numbered in order of appearance. We are also attaching a colorcoded document with the same information in case the mapping between your comments and our responses is not clear.
1) Based on our experience with prior work on MCbased estimation, we do not expect
the linearity (or lack thereof) of the model solution to affect the performance of the
MFMC estimation procedure.2) A goal of MFMC estimation is to achieve (using very few samples of the highfidelity
model) the same accuracy as obtained by an MC estimator (that exclusively uses many
samples of that model). Certainly, at least one highfidelity model evaluation is
necessary to eliminate bias in the MFMC estimator. Moreover, we find practically that
the highfidelity samples used to “steer” this estimator in an accurate direction, while
the lowfidelity samples are used to shrink its variance around the true solution. Note
that the dynamics of the example systems are not particularly well resolved by the low
fidelity models relative to the highfidelity ones; particularly in the case of the
barotropic gyre (SOMA) case, there is a noticeable visual difference between the 32km
solution and the 8km solution.3) This has now been corrected. Thank you for your attention.
4) These were the wallclock times observed when the relevant system was implemented in MATLAB and run on a 2015 MacBook laptop. Therefore, neither the implementation nor the hardware was optimized for computational efficiency.
5) A primary benefit of MFMC over other modern estimation methods such as MCMC is its ability to leverage lowfidelity information to effect costsavings without sacrificing estimator accuracy. In our experience with MFMC in other settings, this benefit translates to much larger cost savings for a given accuracy tolerance when compared to MCMC as well as other MCrelated sampling schemes (e.g., variance reduction MC, importance sampling).
Minor comments:
1) We agree with this reasoning and have changed the name globally throughout the manuscript.
2) This has been fixed, thank you for your attention.
3) It can be shown that the MFMC method presented here is the provably optimal solution (up to rounding) to a particular constrained optimization problem (see Gruber et al 2022, “A multifidelity Monte Carlo method for realistic computational budgets”, for a formal statement). Therefore, we do not think it is a stretch to say there is “no guesswork involved” in this context.
4, 5) These have been fixed, thank you for your attention.
6) This is an expression for the fluid thickness. We have clarified this globally throughout the manuscript.
7) This is true. The referenced preprint (to appear in J. Sci. Comput.) establishes the particular MFMC algorithm which has been applied to climaterelated examples in this paper. Therefore, we refer interested readers to that manuscript for a more detailed description of the MFMC method. Conversely, we write in that preprint that “Forthcoming work will investigate applications of the present MFMC method to complex systems governed by partial differential equations, particularly in the context of climate modeling.”, and provide an empty citation with title and relevant authors. The mentioned work has since become this GMD submission. Since this has caused confusion, we intend to remove the offending citation from the J. Sci. Comput. paper during the proofing stage.
Thanks again, and please let us know if you have other questions.Best,
Anthony, Max, Lili, Rihui, and Zhu
Interactive discussion
Status: closed

RC1: 'Comment on egusphere2022797', Anonymous Referee #1, 09 Oct 2022
Review Report on egusphere2022797
"Multifidelity Monte Carlo Estimation for Efficient Uncertainty Quantification in ClimateRelated Modeling"
by Anthony Gruber, Max Gunzburger, Lili Ju, Rihui Lan, and Zhu Wang
In this paper, the authors extensively tested the multifidelity Monte Carlo (MFMC) method for accurately catching some quantities of interest in climaterelated modeling. Compared to the widely used Monte Carlo (MC) method, they demonstrated that MFMC has superior properties over MC on this subject via three benchmark problems in the ocean, atmosphere, and ice sheet modeling. Since uncertainties exist almost everywhere in climate modeling and prediction, I think MFMC would greatly impact this area, especially in reducing the number of costly largescale simulations in very fine resolutions. Hence, I would highly recommend the publication of this manuscript in GMD after the authors address the following minor issues.
 In line 67 on page 3, the statement is somehow confusing. The authors declare that they don't consider any possible alternate sampling schemes at the beginning. But their goal is to use a nontraditional MC sampling strategy. Doesn't the sampling strategy belong to the "alternate sampling schemes"?
 In Algorithm A, it mentions that F^{k} is removed from the list of surrogates if the second requirement is not satisfied. Will there be such a case all F^{k} will be removed from the estimation? Secondly, will it obtain less accuracy compared to the MC method with all F^{k }included?
 For Figures 3 and 6, it is better to point out that the blue shading is the standard deviation of the MC prediction and the melon for MFMC. Moreover, being included in the blue shading indicates a smaller standard deviation.
Citation: https://doi.org/10.5194/egusphere2022797RC1 
AC1: 'Reply on RC1', Anthony Gruber, 10 Oct 2022
Dear Anonymous Referee #1,Thank you for your helpful comments and for your interest in our work. Below are responses to your specific questions:1) By “alternate sampling strategies” we mean that a user might be using (or may want to use) some other sampling scheme besides traditional MC for the truth model, e.g., Latin hypercube sampling, quasiMC sampling, or sparse grid sampling, as doing so often results in lower costs for a given accuracy tolerance. Therefore, the choice of sampling scheme is determined by how we want to sample the truth model and its surrogates. While we use MC sampling in this work, other choices are possible and would produce analogues to the MFMC method, e.g., MFLH, MFQMC, or MFSG.
2) Note that all F^{k }will never be removed according to our criterion; in the most extreme case, at least the highfidelity model F^{1} will remain. On the other hand, if this occurs, we obtain the useful information that none of the surrogates F^{2}, …, F^{k} are effective at increasing estimation accuracy or reducing computational costs, i.e., we are stuck using MC on its own and should look for other, more informative, surrogates. To address the other question, there should never be a loss of accuracy relative to MC regardless of the model set. The worstcase scenario is if all the surrogates fail the second test, in which case we are left with traditional MC using F^{1}. Hence, there is no loss of accuracy in MFMC; its worstcase accuracy is simply equal to that of MC using the truth model. However, if at least one surrogate survives the second test, then the MFMC method guarantees that costs are reduced without compromising accuracy.
3) We agree that it is a good idea to mention this explicitly. Any future versions of the manuscript will include this change.Thanks again, and please let us know if you have other questions.Best,
Anthony, Max, Lili, Rihui, and ZhuCitation: https://doi.org/10.5194/egusphere2022797AC1

RC2: 'Comment on egusphere2022797', Anonymous Referee #2, 31 Oct 2022
Review for Multifidelity Monte Carlo Estimation for Efficient Uncertainty Quantification in ClimateRelated Modeling
The authors describe a "multifidelity" Monte Carlo method for expected value estimation that is shown to improve convergence/efficiency compared to traditional Monte Carlo methods. This multifidelity approach is different to existing techniques in that expected values are computed using simulations run at differing resolutions and/or incorporating various levels of dynamical approximation. The test cases presented here employ hierarchies of progressively coarsened meshes to provide lowcost estimators, as well as an interpolationbased approach. The authors present results for two shallowwater benchmarks and one icesheet testcase, and demonstrate significant speedups compared to the traditional MC scheme for various maximumvalue estimation problems. The speedups demonstrated for the MFMC method are very attractive, especially in the context of climatemodel estimators which are often extremely expensive to obtain. There are a number of questions I have relating MFMC performance to problem nonlinearity though, in addition to various minor corrections.
Comments: The three testcases analysed appear to involve estimators evaluated after relatively short integration periods, and, rather than fully nonlinear flows/dynamics, the response of the system may be relatively linear over these short horizons as a result  is this expected to influence the effectiveness of the MFMC results presented? For example, TC5 from Williamson does not become strongly nonlinear until approx. 20days, with 50days being the typical analysis window at which turbulence is fully developed. The SOMA case described in Wolfram (2015) is typically spunup over several years, and then analysed over 30day windows. In this work, it appears the TC5 case is analysed after 10 days, and the barotropicgyreversionofSOMA after 3days (restarted from a 15day spinup). Are the MFMC results robust when the duration/nonlinearity of the testcases is increased?
 Is it possible to estimate the relative "multifidelity" contributions to the accuracy of the MFMC estimator? For example, is the overall accuracy governed more by the small number of high resolution runs, the large number of low resolution runs, or something inbetween? Considering the more linear (or at least nonturbulent) nature of solutions studied, how would a conventional MC estimator compare if run only using lowerresolution simulations? In other words, is the good performance of the MFMC method due to the solution being wellresolved even on the coarser meshes, or is the multiresolution hierarchy effective in estimating behaviour resolvable only at highresolution? If it is the former, I wonder whether the problems studied are sufficiently nonlinear at the gridscale. If it is the latter, this may be a nice result to highlight further.
 I believe the gradient terms in the shallowwater system (12) should be grad(1/2*u^2) + g*grad(h + h_b) rather than the grad(rho) included currently. Here p = rho_0*g*h is used to simplify the linear 1/rho_0 * grad(p) shallowwater pressure gradient, consistent with e.g. Ringler et al (2010).
 The SWE runtimes noted in 3.1.1 and 3.2.1 appear to be quite slow  requiring 100's of seconds to advance a single timestep using relatively small O(<= 100,000) cell meshes? Are these runtimes for the full multiday simulations instead, or for all ensemble members perhaps?
 While the MFMC methods presented here are clearly different in that they leverage varying resolution simulations, is it fair to compare against only the "historical" MC method, which is known to be uncompetitive in terms of efficiency? Significant work on alternative MC methods has been conducted by various authors in which a variety of accelerated techniques have been proposed. Are the large gains reported for MFMC expected to be replicated compared to e.g. MCMC (Markov Chain Monte Carlo) approaches more frequently used in climate model estimation?
Minor comments: The SOMA test case (Simulating Ocean Mesoscale Activity) typically refers to simulations using the multilayer primitive equations, in which mesoscale eddies form due to 3d interactions between the momentum, density and forcing tendencies. In this shallowwater configuration with rho = const., it appears to be a winddriven barotropic gyre that's studied instead, which is typically less turbulent, as per the smooth flow features in fig. 2. If so, it's suggested to label this test case as a winddriven gyre.
 Wallis (2012) reference appears to be missing.
 ln 76: Is saying "no guesswork involved" too strong a statement? The systematic nature of the MFMC approach is attractive, but is it *the* provably optimal sampling strategy, or more of an effective heuristic?
 ln 72: ...also uses cheaper to obtain...
 fig. 3 labelling: leftright vs topbottom.
 ln 308: Is this an expression for the free surface height or the layer thickness  h appears to be thickness in the shallowwater system (12).
 The Gruber (2022) paper referenced here appears to be an arXiv preprint, that inturn references this GMD submission??
Citation: https://doi.org/10.5194/egusphere2022797RC2 
AC2: 'Response to Anonymous Referee #2', Anthony Gruber, 11 Nov 2022
Dear Anonymous Referee #2,
Thank you for your helpful comments and for your interest in our work. Below are responses to your specific comments, numbered in order of appearance. We are also attaching a colorcoded document with the same information in case the mapping between your comments and our responses is not clear.
1) Based on our experience with prior work on MCbased estimation, we do not expect
the linearity (or lack thereof) of the model solution to affect the performance of the
MFMC estimation procedure.2) A goal of MFMC estimation is to achieve (using very few samples of the highfidelity
model) the same accuracy as obtained by an MC estimator (that exclusively uses many
samples of that model). Certainly, at least one highfidelity model evaluation is
necessary to eliminate bias in the MFMC estimator. Moreover, we find practically that
the highfidelity samples used to “steer” this estimator in an accurate direction, while
the lowfidelity samples are used to shrink its variance around the true solution. Note
that the dynamics of the example systems are not particularly well resolved by the low
fidelity models relative to the highfidelity ones; particularly in the case of the
barotropic gyre (SOMA) case, there is a noticeable visual difference between the 32km
solution and the 8km solution.3) This has now been corrected. Thank you for your attention.
4) These were the wallclock times observed when the relevant system was implemented in MATLAB and run on a 2015 MacBook laptop. Therefore, neither the implementation nor the hardware was optimized for computational efficiency.
5) A primary benefit of MFMC over other modern estimation methods such as MCMC is its ability to leverage lowfidelity information to effect costsavings without sacrificing estimator accuracy. In our experience with MFMC in other settings, this benefit translates to much larger cost savings for a given accuracy tolerance when compared to MCMC as well as other MCrelated sampling schemes (e.g., variance reduction MC, importance sampling).
Minor comments:
1) We agree with this reasoning and have changed the name globally throughout the manuscript.
2) This has been fixed, thank you for your attention.
3) It can be shown that the MFMC method presented here is the provably optimal solution (up to rounding) to a particular constrained optimization problem (see Gruber et al 2022, “A multifidelity Monte Carlo method for realistic computational budgets”, for a formal statement). Therefore, we do not think it is a stretch to say there is “no guesswork involved” in this context.
4, 5) These have been fixed, thank you for your attention.
6) This is an expression for the fluid thickness. We have clarified this globally throughout the manuscript.
7) This is true. The referenced preprint (to appear in J. Sci. Comput.) establishes the particular MFMC algorithm which has been applied to climaterelated examples in this paper. Therefore, we refer interested readers to that manuscript for a more detailed description of the MFMC method. Conversely, we write in that preprint that “Forthcoming work will investigate applications of the present MFMC method to complex systems governed by partial differential equations, particularly in the context of climate modeling.”, and provide an empty citation with title and relevant authors. The mentioned work has since become this GMD submission. Since this has caused confusion, we intend to remove the offending citation from the J. Sci. Comput. paper during the proofing stage.
Thanks again, and please let us know if you have other questions.Best,
Anthony, Max, Lili, Rihui, and Zhu

AC2: 'Response to Anonymous Referee #2', Anthony Gruber, 11 Nov 2022

AC2: 'Response to Anonymous Referee #2', Anthony Gruber, 11 Nov 2022
Dear Anonymous Referee #2,
Thank you for your helpful comments and for your interest in our work. Below are responses to your specific comments, numbered in order of appearance. We are also attaching a colorcoded document with the same information in case the mapping between your comments and our responses is not clear.
1) Based on our experience with prior work on MCbased estimation, we do not expect
the linearity (or lack thereof) of the model solution to affect the performance of the
MFMC estimation procedure.2) A goal of MFMC estimation is to achieve (using very few samples of the highfidelity
model) the same accuracy as obtained by an MC estimator (that exclusively uses many
samples of that model). Certainly, at least one highfidelity model evaluation is
necessary to eliminate bias in the MFMC estimator. Moreover, we find practically that
the highfidelity samples used to “steer” this estimator in an accurate direction, while
the lowfidelity samples are used to shrink its variance around the true solution. Note
that the dynamics of the example systems are not particularly well resolved by the low
fidelity models relative to the highfidelity ones; particularly in the case of the
barotropic gyre (SOMA) case, there is a noticeable visual difference between the 32km
solution and the 8km solution.3) This has now been corrected. Thank you for your attention.
4) These were the wallclock times observed when the relevant system was implemented in MATLAB and run on a 2015 MacBook laptop. Therefore, neither the implementation nor the hardware was optimized for computational efficiency.
5) A primary benefit of MFMC over other modern estimation methods such as MCMC is its ability to leverage lowfidelity information to effect costsavings without sacrificing estimator accuracy. In our experience with MFMC in other settings, this benefit translates to much larger cost savings for a given accuracy tolerance when compared to MCMC as well as other MCrelated sampling schemes (e.g., variance reduction MC, importance sampling).
Minor comments:
1) We agree with this reasoning and have changed the name globally throughout the manuscript.
2) This has been fixed, thank you for your attention.
3) It can be shown that the MFMC method presented here is the provably optimal solution (up to rounding) to a particular constrained optimization problem (see Gruber et al 2022, “A multifidelity Monte Carlo method for realistic computational budgets”, for a formal statement). Therefore, we do not think it is a stretch to say there is “no guesswork involved” in this context.
4, 5) These have been fixed, thank you for your attention.
6) This is an expression for the fluid thickness. We have clarified this globally throughout the manuscript.
7) This is true. The referenced preprint (to appear in J. Sci. Comput.) establishes the particular MFMC algorithm which has been applied to climaterelated examples in this paper. Therefore, we refer interested readers to that manuscript for a more detailed description of the MFMC method. Conversely, we write in that preprint that “Forthcoming work will investigate applications of the present MFMC method to complex systems governed by partial differential equations, particularly in the context of climate modeling.”, and provide an empty citation with title and relevant authors. The mentioned work has since become this GMD submission. Since this has caused confusion, we intend to remove the offending citation from the J. Sci. Comput. paper during the proofing stage.
Thanks again, and please let us know if you have other questions.Best,
Anthony, Max, Lili, Rihui, and Zhu
Peer review completion
Journal article(s) based on this preprint
Anthony Gruber et al.
Data sets
MultifidelityMonteCarlo Anthony Gruber, Max Gunzburger, Lili Ju, Rihui Lan, Zhu Wang https://doi.org/10.5281/zenodo.7071646
Anthony Gruber et al.
Viewed
HTML  XML  Total  BibTeX  EndNote  

414  99  13  526  6  2 
 HTML: 414
 PDF: 99
 XML: 13
 Total: 526
 BibTeX: 6
 EndNote: 2
Viewed (geographical distribution)
Country  #  Views  % 

Total:  0 
HTML:  0 
PDF:  0 
XML:  0 
 1
The requested preprint has a corresponding peerreviewed final revised paper. You are encouraged to refer to the final revised version.
 Preprint
(3762 KB)  Metadata XML