gcc / g ++：编译大文件时出错 [英] gcc/g++: error when compiling large file

问题描述

我有一个自动生成的C ++源文件，大小约40 MB。它主要包括用于一些向量的push_back命令和应该推送的字符串常量。

当我尝试编译这个文件时，g ++退出并说它不能保留足够的虚拟内存（大约3 GB）。 Googling这个问题，我发现使用命令行开关

   -  param ggc-min-expand = 0 --param ggc -min-heapsize = 4096 
  

可以解决问题。

1）这真的是我正在寻找的解决方案吗？

2）还是有更快，更好的（编译需要花费这些选项的年龄）这样做？

p>

Alexander

更新：感谢所有好的想法。我试过大多数。使用数组而不是几个push_back（）操作减少了内存使用，但是由于我试图编译的文件太大了，它仍然崩溃，只是稍后。在某种程度上，这种行为是非常有趣的，因为在这样的设置中没有太多的优化 - GCC做什么后面的场景，花费这么多的内存？ （我使用停用所有优化编译，并得到相同的结果）

我切换到现在的解决方案是从二进制对象文件中读取原始数据，使用 objcopy 从原始文件创建。这是我最初不想做的，因为用更高级语言（在这种情况下是Perl）创建数据结构比在C ++中更方便。

但是，让这个运行在Win32下比预期更复杂。 objcopy似乎以ELF格式生成文件，似乎当我手动设置输出格式为 pe-i386 时，我已经消失了一些问题。目标文件中的符号是以文件名命名的标准，例如。转换文件 inbuilt_training_data.bin 将导致这两个符号：binary_inbuilt_training_data_bin_start和binary_inbuilt_training_data_bin_end。我在网上找到一些教程，声称这些符号应该声明为 extern char _binary_inbuilt_training_data_bin_start; ，但是这似乎不对 - extern char binary_inbuilt_training_data_bin_start;

解决方案

表。例如，而不是这样：

  void f（）{
 a.push_back（one）; 
 a.push_back（two）; 
 a.push_back（three）; 
 // ... 
} 
  

尝试这样做：

  const char * data [] = {
one，
two，
三，
 // ... 
}; 
 
 void f（）{
 for（size_t i = 0; i  a.push_back （data [i]）; 
} 
} 
  

编译器可能会更高效地生成一个大常量数据表，而不是包含许多 push_back（）调用的巨大函数。

I have a auto-generated C++ source file, around 40 MB in size. It largely consists of push_back commands for some vectors and string constants that shall be pushed.

When I try to compile this file, g++ exits and says that it couldn't reserve enough virtual memory (around 3 GB). Googling this problem, I found that using the command line switches

--param ggc-min-expand=0 --param ggc-min-heapsize=4096

may solve the problem. They, however, only seem to work when optimization is turned on.

1) Is this really the solution that I am looking for?

2) Or is there a faster, better (compiling takes ages with these options acitvated) way to do this?

Best wishes,

Alexander

Update: Thanks for all the good ideas. I tried most of them. Using an array instead of several push_back() operations reduced memory usage, but as the file that I was trying to compile was so big, it still crashed, only later. In a way, this behaviour is really interesting, as there is not much to optimize in such a setting -- what does the GCC do behind the scenes that costs so much memory? (I compiled with deactivating all optimizations as well and got the same results)

The solution that I switched to now is reading in the original data from a binary object file that I created from the original file using objcopy. This is what I originally did not want to do, because creating the data structures in a higher-level language (in this case Perl) was more convenient than having to do this in C++.

However, getting this running under Win32 was more complicated than expected. objcopy seems to generate files in the ELF format, and it seems that some of the problems I had disappeared when I manually set the output format to pe-i386. The symbols in the object file are by standard named after the file name, e.g. converting the file inbuilt_training_data.bin would result in these two symbols: binary_inbuilt_training_data_bin_start and binary_inbuilt_training_data_bin_end. I found some tutorials on the web which claim that these symbols should be declared as extern char _binary_inbuilt_training_data_bin_start;, but this does not seem to be right -- only extern char binary_inbuilt_training_data_bin_start; worked for me.

解决方案

You may be better off using a constant data table instead. For example, instead of doing this:

void f() {
    a.push_back("one");
    a.push_back("two");
    a.push_back("three");
    // ...
}

try doing this:

const char *data[] = {
    "one",
    "two",
    "three",
    // ...
};

void f() {
    for (size_t i = 0; i < sizeof(data)/sizeof(data[0]); i++) {
        a.push_back(data[i]);
    }
}

The compiler will likely be much more efficient generating a large constant data table, rather than huge functions containing many push_back() calls.

这篇关于gcc / g ++：编译大文件时出错的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！