有元组的StackOverflowError [英] StackOverflowError with tuple

问题描述

我编写了一个递归函数，用于在julia中获取较大数组中的对象.发生以下错误:

I have written a recursive function for getting objects in larger arrays in julia. The following error occured:

ERROR: LoadError: StackOverflowError:
     in cat_t at abstractarray.jl:831
     in recGetObjChar at /home/user/Desktop/program.jl:1046
     in recGetObjChar at /home/user/Desktop/program.jl:1075 (repeats 9179 times)
     in getImChars at /home/user/Desktop/program.jl:968
     in main at /home/user/Desktop/program.jl:69
     in include at ./boot.jl:261
     in include_from_node1 at ./loading.jl:304
     in process_options at ./client.jl:308
     in _start at ./client.jl:411
    while loading /home/user/Desktop/program.jl, in expression starting on line 78

如果您想看一下代码，我已经打开了一个问题(声明失败，进程中止).在为julia v 0.4调试我的代码之后，更明显的是导致问题的原因.由于一个对象可能是一个对象，因此locObj的数量远远超过9000个条目. 150 x 150大. 这将导致locObj的长度为22500.讲义可以达到多大?如何避免stackoverflow?还有另一种保存我的价值观的方法吗?

If you want to have a look at the code, I have already opened an issue (Assertion failed, process aborted). After debugging my code for julia v 0.4, it is more obvious, what causes the problem. The tupel locObj gets much bigger than 9000 entries, because one object can be e.g. 150 x 150 big. That would result in a length of 22500 for locObj. How big can tupels get, and how can I avoid a stackoverflow? Is there another way to save my values?

推荐答案

正如我所评论的，我认为存在更好的方法来处理大数据数组，并且此答案主要属于您问题的这一部分:

As it's commented, I think better approaches exist to work with big arrays of data, and this answer is mainly belongs to this part of your question:

还有另一种保存我的值的方法吗?

Is there another way to save my values?

我已经准备了一个测试，以显示使用mmap在处理大数据数组时有何帮助，以下两个函数的作用相同:它们创建3 * 10E6 float64的向量，然后对其进行填充，计算sum和打印结果，在第一个(mmaptest())中，已使用内存映射结构存储Vector{Float64}，而第二个(ramtest())在计算机ram上进行工作:

I have prepared a test to show how using mmap is helpful when dealing with big array of data, following functions both do the same thing: they create a vector of 3*10E6 float64, then fill it, calculate sum and print result, in the first one (mmaptest()), a memory-map structure have been used to store Vector{Float64} while second one (ramtest()) do the work on machine ram:

function mmaptest()
  s = open("./tmp/mmap.bin","w+") # tmp folder must exists in pwd() path
  A = Mmap.mmap(s, Vector{Float64}, 3_000_000)
  for j=1:3_000_000
    A[j]=j
  end
  println("sum = $(sum(A))")
  close(s)
end

function ramtest()
  A = Vector{Float64}(3_000_000)
  for j=1:3_000_000
    A[j]=j
  end
  println("sum = $(sum(A))")
end

然后调用了两个函数并计算了内存分配大小:

then both functions have been called and memory allocation size was calculated:

julia> gc(); # => remove old handles to closed stream

julia> @allocated mmaptest()
  sum = 4.5000015e12
  861684

julia> @allocated ramtest()
  sum = 4.5000015e12
  24072791

从这些测试中可以明显看出，使用内存映射对象，内存分配要小得多.

It's obvious from those tests that with a memory-map object, memory allocation is much smaller.

julia> gc()

julia> @time ramtest()
  sum = 4.5000015e12
  0.012584 seconds (29 allocations: 22.889 MB, 3.43% gc time)

julia> @time mmaptest()
  sum = 4.5000015e12
  0.019602 seconds (58 allocations: 2.277 KB)

从@time测试中可以明显看出，

使用mmap可以使代码变慢，同时需要更少的内存.

as it's clear from @time test, using mmap makes the code slower while needs less memory.

我希望它能对您有所帮助.

I wish it helps you, regards.

这篇关于有元组的StackOverflowError的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！