需要做多少模拟? [英] How many simulations need to do?

问题描述

您好，我的问题与模型验证有关.我已经在 netlogo 中完成了一个程序，我将在我的论文报告中使用它，但现在的问题是，我需要做多少次重复(模拟)来证明我的结果?我已经阅读了一些使用统计方法的方法，我的同事向我推荐了一些很好的数学运算，但我也想从从事计算模型工作的人那里知道过去知道什么样的统计测试或数学方法.

Hello my problem is more related with the validation of a model. I have done a program in netlogo that i'm gonna use in a report for my thesis but now the question is, how many repetitions (simulations) i need to do for justify my results? I already have read some methods using statistical approach and my colleagues have suggested me some nice mathematical operations, but i also want to know from people who works with computational models what kind of statistical test or mathematical method used to know that.

推荐答案

这有两个方面 (1) 有多少参数组合 (2) 每个参数组合运行多少次.

There are two aspects to this (1) How many parameter combinations (2) How many runs for each parameter combination.

(1) 通常你会做实验，改变一些输入参数值，看看一些模型输出如何变化.以众所周知的 Schelling 偏析模型为例，您可以改变容差值并查看偏析指数如何受到影响.在这种情况下，您可以将容差从 0 更改为 1 0.01(如果您想要离散)，或者您可以在 [0,1] 范围内取 100 个不同的随机值.这是一个实验设计的问题，完全取决于您希望检查参数空间的精细程度.

(1) Generally you would do experiments, where you vary some of your input parameter values and see how some model output changes. Take the well known Schelling segregation model as an example, you would vary the tolerance value and see how the segregation index is affected. In this case you might vary the tolerance from 0 to 1 by 0.01 (if you want discrete) or you could just take 100 different random values in the range [0,1]. This is a matter of experimental design and is entirely affected by how fine you wish to examine your parameter space.

(2) 对于每个实验值，您还需要运行多次模拟，以便计算平均值并减少模拟运行中随机性的影响.例如，假设您使用输入参数的值为 3(无论它意味着什么)运行模型并得到 125 的结果.您如何知道真实"答案是 125 还是其他值.如果您运行它 10 次并得到 124.8 到 125.2 范围内的 10 个不同数字，那么 125 并不是一个不合理的估计.如果您运行 10 次并得到 50 到 500 之间的数字，那么 125 就不是一个有用的报告结果.

(2) For each experimental value, you also need to run multiple simulations so that you can can calculate the average and reduce the impact of randomness in the simulation run. For example, say you ran the model with a value of 3 for your input parameter (whatever it means) and got a result of 125. How do you know whether the 'real' answer is 125 or something else. If you ran it 10 times and got 10 different numbers in the range 124.8 to 125.2 then 125 is not an unreasonable estimate. If you ran it 10 times and got numbers ranging from 50 to 500, then 125 is not a useful result to report.

每个实验集的运行次数取决于输出的可变性和您的容忍度.如果您希望能够估计到小数点后 1 位，即使是 124.8 到 125.2 也没有用.在任何统计教科书中查找平均值的标准误差".基本上，如果您进行 N 次运行，那么结果的 95% 置信区间是 N 次运行结果的平均值加/减 1.96 x 结果的标准偏差/sqrt(N).如果您想要更窄的置信区间，则需要更多次运行.

The number of runs for each experiment set depends on the variability of the output and your tolerance. Even the 124.8 to 125.2 is not useful if you want to be able to estimate to 1 decimal place. Look up 'standard error of the mean' in any statistics text book. Basically, if you do N runs, then a 95% confidence interval for the result is the average of the results for your N runs plus/minus 1.96 x standard deviation of the results / sqrt(N). If you want a narrower confidence interval, you need more runs.

另一件需要考虑的事情是，如果您正在寻找参数空间上的关系，那么与尝试对结果进行点估计相比，您在每个点上需要的运行次数更少.

The other thing to consider is that if you are looking for a relationship over the parameter space, then you need fewer runs at each point than if you are trying to do a point estimate of the result.

这篇关于需要做多少模拟?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！