氧气慢 [英] Doxygen is Slow

问题描述

Doxygen在我们的代码库上运行大约需要12个小时.这主要是因为要处理大量代码(约150万行).但是，它很快就会接近我们无法进行每晚文档更新的时间，因为它们花费的时间太长.我们已经不得不减少图表深度以将其减少到12小时.

Doxygen takes about 12 hours to run on our code base. This is primarily because there is a lot of code to process (~1.5M lines). However, it's very quickly approaching the point at which we can't do nightly documentation updates because they take too long. We've already had to reduce the graph depth to get it down to 12 hours.

我已经尝试了标准方法，但是我确实确实需要高质量的输出，其中包括图形和SEARCH_INCLUDES.我有一台相当不错的机器来运行Doxygen，但是Doxygen没有利用其许多内核. (它仅在构建服务器上固定一个CPU，但仅占可用系统的4％.)拥有多线程的Dot构建非常方便，尽管这仅占构建时间的一半左右.

I've tried the standard approaches, but I really do need high quality output, and this includes graphs and SEARCH_INCLUDES. I have a fairly good machine to run Doxygen on, but Doxygen doesn't take advantage of its many cores. (It pegs a single CPU on the build server, but is only 4% of the available system.) Having a multithreaded Dot build is handy, though that's only half or so of the build time.

我是否可以使用多种技术通过多个进程运行doxygen并手动分片任务?我已经看到过一些有关创建标记文件的讨论，但是我对它们的了解还不足以知道它们是否会执行我想要的操作.我正在寻找的东西是这样的:

Are there any techniques I can use to run doxygen via multiple processes and manually shard the task? I've seen some talk about creating tag files, but I don't understand enough about them to know if they'd do what I want. What I'm looking for is something like:

doxygen Doxyfile-folder1
doxygen Doxyfile-folder2
doxygen Doxyfile-folder3
doxygen Doxyfile-folder4
doxygen-join output/folder1/html output/folder2/html output/folder3/html output/folder4/html

当然，我只是在编造东西，但这是我所寻找的想法.另外，我将使用4个以上的进程.

Of course, I'm just making stuff up, but that's an idea of what I'm looking for. Also, I'd use a lot more than 4 processes.

推荐答案

如果

1. 您有许多逻辑上一致的源文件(我们称它们为组件)，并且
2. 您知道组件之间的依赖关系，例如组件A使用组件B和C，而组件B仅使用C，并且
3. 将索引文件(例如文件/类/函数的列表)限制为单个组件是可以的(甚至是首选).
4. 您对HTML输出感兴趣.

标签文件基本上只是符号的结构化列表，带有指向文档中位置的链接.标记文件允许doxygen从一个组件的文档链接到另一组件的文档.

A tag file is basically just a structured list of symbols with links to the location in the documentation. Tag files allow doxygen to make links from the documentation of one component to that of another.

这是一个两步过程:

1. 首先，在每个组件上运行doxygen以生成该组件的标记文件.您可以通过禁用所有输出并使用GENERATE_TAGFILE来做到这一点.因此，对于组件A，Doxyfile.tagonly将具有以下设置:

1. First you run doxygen on each component to generate the tag file for that component. You can do this by disabling all output and use GENERATE_TAGFILE. So for component A, a Doxyfile.tagonly would have the following settings:

GENERATE_HTML         = NO
GENERATE_LATEX        = NO
GENERATE_RTF          = NO
GENERATE_MAN          = NO
GENERATE_TAGFILE      = compA.tag

您会注意到以这种方式运行doxygen的速度非常快.

You'll notice that running doxygen this way is very fast.

第二步是生成实际文档.对于组件A，您需要一个Doxyfile，其中包含组件B和C的标记文件，因为我们确定A依赖于这些组件.

The second step is to generate the actual documentation. For component A you need a Doxyfile which includes the tag files of the components B and C since we determined A depends on these components.

GENERATE_HTML         = YES
GENERATE_LATEX        = NO
GENERATE_RTF          = NO
GENERATE_MAN          = NO
TAGFILES              = path/to/compB/compB.tag=path/to/compB/htmldocs \
                        path/to/compC/compC.tag=path/to/compC/htmldocs

使用这种方法，我已经能够在3小时内在标准台式机(具有8Gb RAM和Linux 64位的Core i5)上为分布在1500+个组件上的2000万行代码生成文档，包括源代码浏览，完整调用图以及所有数据结构的UML样式图.请注意，第一步仅需10分钟.

Using this approach I have been able to generate documentation for 20M+ lines of code distributed over 1500+ components in under 3 hours on a standard desktop PC (Core i5 with 8Gb RAM and Linux 64bit), including source browsing, full call graphs, and UML-style diagrams of all data structures. Note that the first step only took 10 minutes.

为此，我制作了一个脚本，根据组件列表及其直接依赖关系为每个组件生成Doxyfile.第一步，我并行运行8个doxygen实例(使用 http://www.gnu.org/s/parallel/).在第二步中，我并行运行4个doxygen实例.

To accomplish this I made a script to generate the Doxyfile's for each component based on the list of components and their direct dependencies. In the first step I run 8 instances of doxygen in parallel (using http://www.gnu.org/s/parallel/). In the second step I run 4 instances of doxygen in parallel.

请参见 http://www.doxygen.nl/manual/external.html 有关标签文件的更多信息.

See http://www.doxygen.nl/manual/external.html for more info about tag files.

这篇关于氧气慢的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！