为什么当我使用 SOLR 查询所有文档时 CPU 使用率接近 100% [英] why CPU usage close to reach to 100% when i use SOLR query all document

问题描述

我有一个应用程序使用 SOLR 来查询 200 万多个文档并按时间排序.查询URL参数像这样/select?sort=p_review_date desc&rows=10&start=0&q=*:*，参数start为变量值，每次请求增加10.做性能压力测试时，SOLR服务器CPU使用率接近100%.

I have a application using SOLR to query 2 million+ document and sort by time. Query URL param like this /select?sort=p_review_date desc&rows=10&start=0&q=*:*, parameter start is variable value, every request increased 10. When i make performance stress testing, the SOLR server CPU usage close to reach to 100%.

问题:1.CPU使用率高的原因是什么?2.有没有办法降低CPU使用率，比如缓存或者其他配置.

Question: 1.What causes high CPU usage? 2.Is there a way to make low CPU usage, such as cache or other configuration.

这是solrconfig.xml中查询缓存配置的一部分1024

This is a part section for query cache configuration in solrconfig.xml 1024

<queryResultCache
    class="solr.LRUCache"
    size="40960"
    initialSize="10240"
    autowarmCount="512"/>

<documentCache
    class="solr.FastLRUCache"
    size="40960"
    initialSize="10240"
    autowarmCount="0"/>

<enableLazyFieldLoading>true</enableLazyFieldLoading>

<queryResultWindowSize>10</queryResultWindowSize>

<queryResultMaxDocsCached>500</queryResultMaxDocsCached>

推荐答案

我在管理大型索引(使用 Autonomy 搜索引擎)方面的经验是，100% 的 CPU 是可以预期的.由于您的索引存储在 RAM 中，因此没有 I/O 活动来减慢搜索操作的速度，CPU 将尝试尽可能快地读取 RAM，这意味着接近 100% CPU.

My experience with managing a large index like you have (using Autonomy search engine), is that 100% CPU is to be expected. Because your index is stored in RAM, there is no I/O activity to slow down the search operation, and CPU will try to read through the RAM as fast as possible, meaning close to 100% CPU.

为什么您希望您的搜索使用 50% 的 CPU，它会花费 2 倍的时间，对吗?

Why would you want your search to use 50% CPU, it will take 2x as long, right?

有一个用于 AIX 的性能监控工具，称为 nmon.在nmonFAQ中，作者提醒我们

There is a performance monitoring tool for AIX called nmon. In the FAQ for nmon, the author reminds us

如果你一直使用越来越短的你最终会看到CPU 要么 100% 忙，要么 100%空闲所有其他数字只是一个人类思维不快的特征足够了，不得不平均CPU 使用时间更长.

If you keep using shorter and shorter periods you will eventually see that the CPUs are either 100% busy or 100% idle all the other numbers are just a feature of humans not thinking fast enough and having to average out the CPU use in longer periods.

即使您没有使用 AIX 作为您的操作系统，nmon FAQ 中也有很多关于系统/应用程序性能监控和测量的好信息.我推荐它，或者您通过谷歌搜索更多关于如何诊断系统性能问题的信息.

Even if you are not using AIX as your OS, there is a lot of good information in the nmon FAQ about system/application performance monitoring and measurement. I recommend it, or that you google more for how to diagnose performance issues in your system.

此外，您可能希望在您的帖子中添加一些标签.SO 有大量的基准测试和测试活动.

Also, you may want to add some tags to your posting. There is a fair amount of activity here at SO for bench-marking and testing.

我希望这会有所帮助.

这篇关于为什么当我使用 SOLR 查询所有文档时 CPU 使用率接近 100%的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！