Doxygen 很慢 [英] Doxygen is Slow

问题描述

Doxygen 在我们的代码库上运行大约需要 12 个小时.这主要是因为要处理大量代码(约 150 万行).但是，它很快就会接近我们无法进行夜间文档更新的地步，因为它们花费的时间太长.我们已经不得不减少图形深度以将其缩短到 12 小时.

Doxygen takes about 12 hours to run on our code base. This is primarily because there is a lot of code to process (~1.5M lines). However, it's very quickly approaching the point at which we can't do nightly documentation updates because they take too long. We've already had to reduce the graph depth to get it down to 12 hours.

我已经尝试过标准方法，但我确实需要高质量的输出，这包括图表和 SEARCH_INCLUDES.我有一台相当不错的机器来运行 Doxygen，但 Doxygen 并没有利用它的许多内核.(它在构建服务器上固定一个 CPU，但仅占可用系统的 4%.)拥有多线程 Dot 构建很方便，尽管这只是构建时间的一半左右.

I've tried the standard approaches, but I really do need high quality output, and this includes graphs and SEARCH_INCLUDES. I have a fairly good machine to run Doxygen on, but Doxygen doesn't take advantage of its many cores. (It pegs a single CPU on the build server, but is only 4% of the available system.) Having a multithreaded Dot build is handy, though that's only half or so of the build time.

有什么技术可以让我通过多个进程运行 doxygen 并手动分片任务吗?我看过一些关于创建标签文件的讨论，但我对它们的了解还不够，不知道它们是否会做我想做的事.我正在寻找的是这样的:

Are there any techniques I can use to run doxygen via multiple processes and manually shard the task? I've seen some talk about creating tag files, but I don't understand enough about them to know if they'd do what I want. What I'm looking for is something like:

doxygen Doxyfile-folder1
doxygen Doxyfile-folder2
doxygen Doxyfile-folder3
doxygen Doxyfile-folder4
doxygen-join output/folder1/html output/folder2/html output/folder3/html output/folder4/html

当然，我只是在编造一些东西，但这是我要找的东西的一个想法.另外，我会使用超过 4 个进程.

Of course, I'm just making stuff up, but that's an idea of what I'm looking for. Also, I'd use a lot more than 4 processes.

推荐答案

标签文件通常是要走的路

Tag files are typically the way to go if

1. 您有许多逻辑上一致的源文件(我们称它们为组件)和
2. 您知道组件之间的依赖关系，例如组件 A 使用组件 B 和 C，组件 B 仅使用 C，并且
3. 将索引文件(例如文件/类/函数的列表)限制为单个组件是可以的(甚至是首选).
4. 您对 HTML 输出感兴趣.

标签文件基本上只是一个结构化的符号列表，其中包含指向文档中位置的链接.标签文件允许 doxygen 将一个组件的文档链接到另一个组件的文档.

A tag file is basically just a structured list of symbols with links to the location in the documentation. Tag files allow doxygen to make links from the documentation of one component to that of another.

这是一个两步过程:

1. 首先在每个组件上运行 doxygen 以生成该组件的标记文件.您可以通过禁用所有输出并使用 GENERATE_TAGFILE 来做到这一点.因此对于组件 A，Doxyfile.tagonly 将具有以下设置:

1. First you run doxygen on each component to generate the tag file for that component. You can do this by disabling all output and use GENERATE_TAGFILE. So for component A, a Doxyfile.tagonly would have the following settings:

GENERATE_HTML         = NO
GENERATE_LATEX        = NO
GENERATE_RTF          = NO
GENERATE_MAN          = NO
GENERATE_TAGFILE      = compA.tag

您会注意到以这种方式运行 doxygen 非常快.

You'll notice that running doxygen this way is very fast.

第二步是生成实际的文档.对于组件 A，您需要一个包含组件 B 和 C 的标记文件的 Doxyfile，因为我们确定 A 依赖于这些组件.

The second step is to generate the actual documentation. For component A you need a Doxyfile which includes the tag files of the components B and C since we determined A depends on these components.

GENERATE_HTML         = YES
GENERATE_LATEX        = NO
GENERATE_RTF          = NO
GENERATE_MAN          = NO
TAGFILES              = path/to/compB/compB.tag=path/to/compB/htmldocs 
                        path/to/compC/compC.tag=path/to/compC/htmldocs

使用这种方法，我能够在不到 3 小时的时间内在标准台式 PC(具有 8Gb RAM 和 Linux 64 位的 Core i5)上为分布在 1500 多个组件上的 2000 万多行代码生成文档，包括源代码浏览、完整调用图，以及所有数据结构的 UML 样式图.请注意，第一步只用了 10 分钟.

Using this approach I have been able to generate documentation for 20M+ lines of code distributed over 1500+ components in under 3 hours on a standard desktop PC (Core i5 with 8Gb RAM and Linux 64bit), including source browsing, full call graphs, and UML-style diagrams of all data structures. Note that the first step only took 10 minutes.

为了实现这一点，我编写了一个脚本，根据组件列表及其直接依赖关系为每个组件生成 Doxyfile.在第一步中，我并行运行 8 个 doxygen 实例(使用 http://www.gnu.org/s/并行/).在第二步中，我并行运行 4 个 doxygen 实例.

To accomplish this I made a script to generate the Doxyfile's for each component based on the list of components and their direct dependencies. In the first step I run 8 instances of doxygen in parallel (using http://www.gnu.org/s/parallel/). In the second step I run 4 instances of doxygen in parallel.

参见 http://www.doxygen.nl/manual/external.html有关标记文件的更多信息.

See http://www.doxygen.nl/manual/external.html for more info about tag files.

这篇关于Doxygen 很慢的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！