Mathematica内存不足 [英] Mathematica running out of memory

问题描述

我正在尝试运行以下程序，该程序计算系数仅为+1或-1的d级多项式的根，然后将其存储到文件中.

I'm trying to run the following program, which calculates roots of polynomials of degree up to d with coefficients only +1 or -1, and then store it into files.

d = 20; n = 18000; 
f[z_, i_] := Sum[(2 Mod[Floor[(i - 1)/2^k], 2] - 1) z^(d - k), {k, 0, d}];

此处f [z，i]给出z上的多项式，带有正负号的二进制计数.假设d = 2，我们应该有

Here f[z,i] gives a polynomial in z with plus or minus signs counting in binary. Say d=2, we would have

f [z，1] = -z ² -z-1
f [z，2] = -z ² -z + 1
f [z，3] = -z ² + z-1
f [z，4] = -z ² + z + 1

f[z,1] = -z² - z - 1
f[z,2] = -z² - z + 1
f[z,3] = -z² + z - 1
f[z,4] = -z² + z + 1

DistributeDefinitions[d, n, f]

ParallelDo[ 
            Do[ 
                     root = N[Root[f[z, i], j]];
                     {a, b} = Round[n ({Re[root], Im[root]}/1.5 + 1)/2];
            {i, 1, 2^d}],
{j, 1, d}]

我意识到阅读这本书可能不太愉快，但是无论如何它都比较短.我会尽量减少相关的部分，但是在这里我真的不知道麻烦是什么.我正在计算f [z，i]的所有根，然后将它们四舍五入以使其对应于n x n网格中的一个点，并将该数据保存在各种文件中.

I realise reading this probably isn't too enjoyable, but it's relatively short anyway. I would've tried to cut down to the relevant parts, but here I really have no clue what the trouble is. I'm calculating all roots of f[z,i], and then just round them to make them correspond to a point in a n by n grid, and save that data in various files.

由于某些原因，Mathematica中的内存使用量会逐渐增加，直到它填满所有内存(此计算机上为6 GB)为止；然后计算会非常缓慢地进行；为什么会这样?

For some reason, the memory usage in Mathematica creeps up until it fills all the memory (6 GB on this machine); then the computation continues extremely slowly; why is this?

我不确定这里是什么用尽了内存-我唯一的猜测是文件流耗尽了内存，但事实并非如此:我尝试将数据追加到2GB文件中，并且没有明显的内存使用情况.似乎绝对没有理由让Mathematica在这里使用大量内存.

I am not sure what is using up the memory here - my only guess was the stream of files used up memory, but that's not the case: I tried appending data to 2GB files and there was no noticeable memory usage for that. There seems to be absolutely no reason for Mathematica to be using large amounts of memory here.

对于较小的d(例如15)，其行为如下:我有4个内核在运行.当它们全部通过ParallelDo循环运行(每个循环一次执行j的值)时，内存使用量会增加，直到它们全部完成一次遍历该循环为止.然后，下次他们经过该循环时，内存使用量根本不会增加.计算最终完成，一切都很好.

For small values of d (15 for example), the behaviour is the following: I have 4 kernels running. As they all run through the ParallelDo loop (each doing a value of j at a time), the memory use increases, until they all finish going through that loop once. Then the next times they go through that loop, the memory use does not increase at all. The calculation eventually finishes and everything is fine.

同样非常重要的是，一旦计算停止，内存使用就不会减少. 如果我开始另一次计算，则会发生以下情况:

Also, quite importantly, once the calculation stops, the memory use does not go back down. If I start another calculation, the following happens:

-如果先前的计算在内存使用量仍在增加时停止了，那么它会继续增加(可能需要一段时间才能再次开始增加，基本上达到计算中的同一点).

-If the previous calculation stopped when memory use was still increasing, it continues to increase (it might take a while to start increasing again, basically to get to the same point in the computation).

-如果在不增加内存使用量的情况下停止了先前的计算，则它不会进一步增加.

-If the previous calculation stopped when memory use was not increasing, it does not increase further.

问题似乎来自f的相对复杂性-将其更改为一些更简单的多项式似乎可以解决问题.我以为问题可能是Mathematica记得f [z，i]为i的特定值，但设置f [z，i]:=.在计算了f [z，i]的根之后，就抱怨该赋值最初不存在，并且仍在使用内存.

The issue seems to come from the relative complexity of f - changing it into some easier polynomial seems to fix the issue. I thought the problem might be that Mathematica remembers f[z,i] for specific values of i, but setting f[z,i] :=. just after calculating a root of f[z,i] complains that the assignment did not exist in the first place, and the memory is still used.

这确实令人感到困惑，因为f是我能想象的唯一剩下的占用内存的事情，但是在内部Do循环中定义f并在每次计算出根之后都将其清除并不能解决问题.

It's quite puzzling really, as f is the only remaining thing I can imagine taking up memory, but defining f in the inner Do loop and clearing it each time after a root is calculated does not solve the problem.

推荐答案

哦，这真令人讨厌.

这是怎么回事，N将对结果进行缓存，以便在您再次需要它们时加快以后的计算速度.有时这绝对是您想要的，但有时它却打破了世界.幸运的是，您确实有一些选择.一种是使用 ClearSystemCache 命令，该命令的作用与锡.我在无并行循环中运行了一会儿之后(在感到无聊和中止计算之前)，MemoryInUse报告使用了约160 MiB.使用ClearSystemCache可以将其降低到大约14 MiB.

What's going on is that N will do caching of results in order to speed up future calculations if you need them again. Sometimes this is absolutely what you want, but sometimes it just breaks the world. Fortunately, you do have some options. One is to use the ClearSystemCache command, which does just what it said on the tin. After I ran your un-parallelized loop for a little while (before getting bored and aborting the calculation), MemoryInUse reported ~160 MiB in use. Using ClearSystemCache got that down to about 14 MiB.

要做的一件事是使用ClearSystemCache > SetSystemOptions 更改缓存行为.您应该看看SystemOptions["CacheOptions"]看看有什么可能.

One thing you should look at doing, instead of calling ClearSystemCache programmatically, is to use SetSystemOptions to change the caching behavior. You should take a look at SystemOptions["CacheOptions"] to see what the possibilities are.

编辑:缓存会为更复杂的表达式带来更大的问题，这并不令人惊讶.必须将这些表达式的副本存储在某个地方，并且更复杂的表达式需要更多的内存.

It's not terribly surprising that the caching causes a bigger problem for more complex expressions. It's got to be stashing copies of those expressions somewhere, and more complex expressions require more memory.

这篇关于Mathematica内存不足的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！