SonarQube后台任务运行超过4个小时 [英] SonarQube background tasks runs past 4 hour

问题描述

我们正在将SonarQube从4.5.X升级到5.6.X，计算引擎是一个新概念，并且从表面上讲非常灵敏. 但是当它投入工作时，它执行的并不多！如果您成功了，可以帮助我度过难关

We are upgrading our SonarQube from 4.5.X to 5.6.X, compute engine is a new concept and on the paper is pretty shinny. But when put to work, it does not perform much! help me get through if you had made it

我选择了一个用SonarQube 4.5.4在4分钟内扫描的测试项目.现在，使用(渐变)扫描仪进行的初始扫描需要3分钟.在测试项目中，CE后台任务将永远运行，第一个任务在183分钟内完成，其他任务在263分钟内完成，第三个任务仍在运行.

I choose a test projects that was scanned under 4 minutes with SonarQube 4.5.4. Now the initial scan taking place with (gradle) scanner takes 3 minutes. And CE background tasks runs forever, out of the test projects, first one completed in 183 minutes, other in 263 minutes, third one is still running.

很显然，我正在挖掘并看到了这个stackoverflow问题- Sonarqube后台任务永远运行并增加了引用的答案的记忆力，但它对我不起作用.

Obviously I got digging and saw this stackoverflow question - Sonarqube background tasks running forever and increased the memory as quoted as answer, but its is not working for me.

sonar.ce.javaOpts=-Xmx4096m -Xms512m -XX:+HeapDumpOnOutOfMemoryError -Djava.net.preferIPv4Stack=true
sonar.ce.workerCount=5

虽然工作人员人数为5，但我现在没有运行并行任务.从完成的作业日志中，我可以看到一些令人震惊的时间信息！

Though worker count is 5, I'm not running parallel tasks now. From the completed job's logs I'm able to see some alarming time info!

2018.02.13 16:56:07 INFO  [o.s.s.c.s.ComputationStepExecutor] Compute complexity measures | time=335ms
2018.02.13 16:56:07 INFO  [o.s.s.c.s.ComputationStepExecutor] Load measure computers | time=6ms
2018.02.13 21:18:34 INFO  [o.s.s.c.s.ExecuteVisitorsStep]   Execution time for each component visitor:
2018.02.13 21:18:34 INFO  [o.s.s.c.s.ExecuteVisitorsStep]   - LoadComponentUuidsHavingOpenIssuesVisitor | time=2483ms
2018.02.13 21:18:34 INFO  [o.s.s.c.s.ExecuteVisitorsStep]   - IntegrateIssuesVisitor | time=15743879ms
2018.02.13 21:18:34 INFO  [o.s.s.c.s.ExecuteVisitorsStep]   - CloseIssuesOnRemovedComponentsVisitor | time=0ms
2018.02.13 21:18:34 INFO  [o.s.s.c.s.ExecuteVisitorsStep]   - QualityModelMeasuresVisitor | time=143ms
2018.02.13 21:18:34 INFO  [o.s.s.c.s.ExecuteVisitorsStep]   - NewQualityModelMeasuresVisitor | time=73ms
2018.02.13 21:18:34 INFO  [o.s.s.c.s.ExecuteVisitorsStep]   - LastCommitVisitor | time=4ms
2018.02.13 21:18:34 INFO  [o.s.s.c.s.ExecuteVisitorsStep]   - MeasureComputersVisitor | time=25ms
2018.02.13 21:18:34 INFO  [o.s.s.c.s.ComputationStepExecutor] Execute component visitors | time=15746700ms
2018.02.13 21:18:37 INFO  [o.s.s.c.s.ComputationStepExecutor] Compute measure variations | time=3290ms
2018.02.13 21:18:37 INFO  [o.s.s.c.s.ComputationStepExecutor] Compute Quality Gate measures | time=54ms
2018.02.13 21:18:37 INFO  [o.s.s.c.s.ComputationStepExecutor] Compute Quality profile measures | time=25ms
2018.02.13 21:18:38 INFO  [o.s.s.c.s.ComputationStepExecutor] Generate Quality profile events | time=33ms
2018.02.13 21:18:38 INFO  [o.s.s.c.s.ComputationStepExecutor] Generate Quality gate events | time=6ms

执行组件访问者，特别是IntegrateIssuesVisitor花费大量时间，在执行所需的总263分钟中占262分钟. 如何避免这种情况?

Execute component visitor, particularly IntegrateIssuesVisitor takes hell a lot of time, 262 minutes out of total 263 minutes required for execution. How to avoid this?

我看不到任何跨项目分析禁用选项，这与之相关吗?

I don't see any cross project analysis disable option, is it related to that?

其他信息: 我使用Oracle DB

Additional information: I use Oracle DB

[更新]取消了对CE的诅咒.我不好，CE太棒了！现在，一些项目将在3秒内被分析和发布！

推荐答案

已解决！

这与数据库有关，正如Google SonarQube小组的讨论所指出的.

It had to do with Database, as discussions on SonarQube Google groups pointed out.

来自 https://docs.sonarqube.org/display/PLUG/管理+计算+引擎+性能， 显然，Compute Engine的性能与之息息相关，

From https://docs.sonarqube.org/display/PLUG/Managing+Compute+Engine+Performance, It is clear that Compute Engine performance are tied to,

• 数据库
• 磁盘I/O
• 网络
• 堆
• CPU

我们的内存消耗和CPU使用率总是最小的，因此我们立即排除了最后两个.但是我们还不知道出什么问题了，如果因为没有将日志级别应用于CE日志，那么任何CE日志在各个方面都是无用的，那么您只有INFO，DEBUG或TRACE级别.还是有提示，

Our memory consumption and CPU usage were always minimal and so straight away we ruled out last two. But we didn't know what was wrong yet, if anything CE logs are useless in every way since log level is not applied to CE logs, you are stuff with only INFO no DEBUG or TRACE levels. Still we took a hint,

INFO  [o.s.s.c.s.ExecuteVisitorsStep]   Execution time for each component visitor:
INFO  [o.s.s.c.s.ExecuteVisitorsStep]   - LoadComponentUuidsHavingOpenIssuesVisitor | time=58ms
INFO  [o.s.s.c.s.ExecuteVisitorsStep]   - IntegrateIssuesVisitor | time=41147606ms
INFO  [o.s.s.c.s.ExecuteVisitorsStep]   - CloseIssuesOnRemovedComponentsVisitor | time=1ms
INFO  [o.s.s.c.s.ExecuteVisitorsStep]   - QualityModelMeasuresVisitor | time=206ms
INFO  [o.s.s.c.s.ExecuteVisitorsStep]   - NewQualityModelMeasuresVisitor | time=119ms
INFO  [o.s.s.c.s.ExecuteVisitorsStep]   - LastCommitVisitor | time=6ms
INFO  [o.s.s.c.s.ExecuteVisitorsStep]   - MeasureComputersVisitor | time=29336ms

IntegrateIssuesVisitor需要花费大量时间来执行，日志仅是提供信息的原因，而为何该部分仍然是个谜.因此，我们决定研究一下代码，感谢SonarQube是开源的，但很快，DBA团队就给我们提出了为什么我们的查询表现如此糟糕的原因.困住了我们，索引我们分析了Sonar的交易表格，现在很好.

IntegrateIssuesVisitor was taking outrageous amount of time to execute, the log was only that informative and the why part was still mystery. So we decided to dig the code, thank god SonarQube is open source but soon enough or DBA team rang us up why our query is behaving so badly. And it stuck us, indexes we Analyzed the transaction tables of Sonar and we are good now.

tl; dr:使表统计信息保持更新，以便优化始终在您的支持下进行.分析表格.

tl;dr : Keep your table statistics updated so optimization are always working in your favor. Analyze tables.

这篇关于SonarQube后台任务运行超过4个小时的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！