如何减少单个.cpp文件的大型C ++库的编译时间? [英] How to reduce compile time for large C++ library of individual .cpp files?

问题描述

我们正在开发一个 C ++库，其中目前包含500多个单独的.cpp文件.它们都被编译并存档到静态库中.即使使用并行构建，也要花费几分钟.我想减少编译时间.

We're developing a C++ library with currently over 500 hundred individual .cpp files. These are each compiled and archived into a static library. Even with a parallel build, this takes some minutes. I'd like to reduce this compilation time.

每个文件平均有110行，其中有一个或两个函数.但是，对于每个.cpp文件，都有一个对应的.h标头，并且许多.cpp文件中通常都包含这些标头.例如，A.cpp，B.cpp，C.cpp等可能包含A.h.

Each file is on average 110 lines with a function or two inside. However, for each .cpp file there is a corresponding .h header and these are often included by many of the .cpp files. For example, A.h might be included by A.cpp, B.cpp, C.cpp, and so on.

我首先想配置编译过程.有没有办法找出花费多少时间做某事?我担心很多时间浪费在打开头文件上，而只是检查头文件的保护措施而忽略该文件.

I'd first like to profile the compilation process. Is there a way to find out how much time is spent doing what? I'm worried that a lot of time is wasted opening header files only to check the include guards and ignore the file.

如果是这种情况的罪魁祸首，那么减少编译时间的最佳实践是什么?

If that sort of thing is the culprit, what are best practices for reducing compilation time?

我愿意添加新的分组头，但可能不愿意更改此多文件布局，因为这使我们的库也可以用作按需仅头文件的库./p>

I'm willing to add new grouping headers, but probably not willing to change this many-file layout since this allows our library to also function as an as-needed header-only library.

推荐答案

这很难说.

我致力于改善工作中的项目的编译时间，发现一个文件花费了15分钟(在-O2中进行编译，而在-O0中则花费了15秒)，并且被编译了两次，所以总的来说编译时间约为60-70分钟，这大约是一半的时间.关闭ONE优化功能后，一个文件的时间减少了大约20秒，而不是15分钟...这个文件产生的是一个机器生成的函数，它的长度为几万行，这使编译器做了一些魔术长的东西(大概是O(N ^ 2)算法).

I worked on improving the compile time on our project at work, and found that ONE file took 15 minutes (when compiling in -O2, but about 15 seconds in -O0) and gets compiled twice, so for a total compile time of about 60-70 minutes, this was roughly half the time. Turning off ONE optimisation feature brought that one file down to about 20 seconds instead of 15 minutes... This file was producing one function that was machine-generated and a few tens of thousands of lines long, which cause the compiler to do some magic long stuff (presumably some O(N^2) algorithm).

如果您有一个小的函数，然后依次调用许多小的函数，最终又通过内联的层变成一个大文件，也会发生这种情况.

This can also happen if you have a small function that then calls lots of small functions in turn, that eventually, through layers of inlining, turns into a large file.

在其他时候，我发现减少文件数量并在一个文件中放置更多代码会更好.

At other times, I've found that reducing the number of files and putting more code in one file works better.

通常，我的经验(包括我自己的编译器项目以及其他人/公司的编译器)都是花时间而不是解析和读取文件，而是经过各种优化和代码生成过程.您可以尝试使用-fsyntax-only或编译器调用的所有文件来编译所有文件.只需阅读源代码并检查其语法正确性即可.如果还没有，请尝试使用-O0进行编译.通常，特定的优化过程是问题所在，有些过程比其他过程更糟，因此检查特定的-O选项中有哪些单独的优化过程很有用-在gcc中可以用-Q -O2 --help=optimizers列出[在这种情况下对于-O2].

In general, my experience (both with my own compiler project, and other people's/company's compilers) is that it's NOT the parsing and reading of files that take the time, but the various optimisation and code-generation passes. You can try that out by compiling all files using -fsyntax-only or whatever it is called for your compiler. That will JUST read the source and check that it's syntactically correct. Try also compiling with -O0 if you aren't already. Often a specific optimisation pass is the problem, and some passes are worse than others, so it's useful to check what individual optimisation passes there are in a particular -O option - in gcc that can be listed with -Q -O2 --help=optimizers [in this case for -O2].

您确实需要弄清楚编译器在花时间在做什么.如果问题是您花费了大部分时间来优化代码，那么更改代码就没有意义了.如果花费时间在解析上，则减少优化器是没有意义的，并且优化不会增加额外的时间.没有实际构建您的项目，很难确定.

You really need to figure out what the compiler is spending time on. It's no point in changing the code around if the problem is that you are spending most of the time optimising the code. It's no point in cutting down optimisers if the time is spent in parsing, and optimisation adds no extra time. Without actually building YOUR project, it's very hard to say for sure.

另一个提示是检查top以查看您的编译进程是否每个使用100％cpu-如果不使用，则您的编译机可能没有足够的内存.我为我的工作项目提供了一个生成选项，该选项通过耗尽大量内存来杀死"我的台式机，整个系统只是陷入停顿-甚至从Web浏览器中的一个选项卡切换到另一个选项卡都需要15到30秒.唯一的解决方案是减少运行-j [但是，我通常会忘记这一点，因此-如果我不想打断它，我会去吃午饭，喝咖啡或其他类似的事情，直到完成为止，因为该机器无法使用].这仅用于调试版本，因为将大型代码库的调试信息汇总在一起会占用大量内存(显然！)

Another tip is to check top to see if your compile processes uses 100% cpu each - if not, you're probably not having enough memory in your compile machine. I have a build option for my work project which "kills" my desktop machine by running so much out of memory the whole system just grinds to a halt - even switching from one tab to another in the web-browser takes 15-30 seconds. The only solution is to run less -j [but of course, I usually forget, and at that point - so if I don't want to interrupt it, I go for lunch, long coffee break or some such until it finishes, because the machine is just unusuable]. This is for debug builds only, because putting together the debug info for the large codebase takes up a lot of memory [apparently!]

这篇关于如何减少单个.cpp文件的大型C ++库的编译时间?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！