一旦我的应用程序爬升到 >，Sunspot-Solr 就会放慢速度1000 个对象 [包括 Solr 日志] [英] Sunspot-Solr slowing down to a beast once my Application climbed to > 1000 objects [ Solr Logs Included ]
问题描述
我很好奇是否有人注意到 Sunspot-Solr 的任何缩放问题.即使我删除了所有可搜索的参数，它也只是自己计算原始类；在我的本地加载仍然需要 5 到 8 秒，在生产中需要 4 到 5 秒.
I'm curious if anyone noticed any scaling issues with Sunspot-Solr . Even if I remove all the searchable params, and its just counting against the raw class by itself; it still takes 5 to 8 seconds to load on my local, 4 to 5 seconds on production.
有其他人能够扩展 Sunspot-Solr 吗?有哪些常见问题?
Has anyone else been able to scale Sunspot-Solr ? What are some common problems?
如何更深入地了解这一点?
How can one look into this deeper?
这是单个请求的 Solr 日志:
Here's the Solr Logs for a single request :
Solr Select (208.1ms) {:rows=>20, :start=>0, :q=>"*:*", :sort=>"score desc", :fq=>["type:Organization", "published_b:true", "updated_at_d:[2009\-02\-03T16\:11\:55Z TO *]"]}
Solr Select (5.5ms) {:rows=>20, :start=>0, :q=>"*:*", :sort=>"score desc", :fq=>["type:Organization", "published_b:true", "updated_at_d:[2009\-02\-03T16\:11\:55Z TO *]"]}
Solr Update (12.6ms) <?xml version="1.0" encoding="UTF-8"?><add><doc><field name="type">User</field><field name="type">ActiveRecord::Base</field><field name="id">User 2</field><field name="class_name">User</field><field name="first_name_s">Bob</field><field name="created_at_d">2009-09-28T21:00:27Z</field><field name="last_name_s">Marley</field><field name="email_s">bob.marley@gmail.com</field><field name="name_s">Bob Marley</field><field name="last_name_text">Marley</field><field name="first_name_text">Bob</field><field name="email_text">bob.marley@gmail.com</field><field name="name_text">Bob Marley</field></doc></add>
Solr Update (487.7ms) <?xml version="1.0" encoding="UTF-8"?><commit/>
Completed in 12632ms (View: 11633, DB: 228) | 200 OK [http://localhost/organizations/search]
推荐答案
1000 个对象对于 Solr 来说是小菜一碟，所以在大约 200 毫秒的 Solr 读取时，这里发生了一些可疑的事情.但是，您最直接的问题是您在看似是 GET 请求的过程中向 Solr 写信——这是怎么回事?您是否正在保存触发 Sunspot 自动索引的可搜索对象?如果您需要在 GET 请求过程中更新模型(如果可能，应该在后台作业中完成)，您需要在 Sunspot 中禁用自动索引:
1000 objects is child's play for Solr, so there's something fishy going on here with the ~200ms Solr read. However, your most immediate problem is that you're writing to Solr during what appears to be a GET request -- what's up with that? Are you saving a searchable object, which is triggering Sunspot's auto-index? If you have the need to update models during the course of a GET request (which should probably be done in a background job if possible), you'll want to disable auto-indexing in Sunspot:
searchable :auto_index => false
# sunspot setup
end
然后，当您确实想在 Solr 中更新它们时，您需要在控制器中显式调用 my_model.index.
Then you'd need to explicitly call my_model.index in your controllers when you actually do want to update them in Solr.
最后，最后的重大更新是 Solr 提交，它告诉 Solr 将未暂存的更改写入磁盘并加载反映这些更改的新搜索器.提交是昂贵的；Sunspot::Rails 默认在写入 Solr 的任何请求结束时执行提交，但这种行为的目标是为 Sunspot 的新用户提供最少惊喜的原则，而不是生产中的实时应用程序.您需要在 config/sunspot.yml 中禁用它:
Finally, that big update at the end is a Solr commit, which tells Solr to write unstaged changes to disk and load up a new searcher that reflects those changes. Commits are expensive; Sunspot::Rails by default performs a commit at the end of any request that writes to Solr, but this behavior is targeted at the principle of least surprise for new users of Sunspot rather than a live app in production. You'll want to disable it in your config/sunspot.yml:
auto_commit_after_request: false
然后您可能希望在您的 solr/conf/solrconfig.xml 中配置 autoCommit -- 它在默认的 Sunspot Solr 发行版中被注释掉，并且那里也有解释.我发现每分钟一次是一个很好的起点.
You'll then probably want to configure autoCommit in your solr/conf/solrconfig.xml -- it's commented out in the default Sunspot Solr distribution, and there is an explanation in there too. I've found that once a minute is a good place to start.
进行这些更改后，我会看看您的读取是否仍然很慢——我认为很可能是因为每次您搜索时，您对 Solr 的写入/提交导致它必须从磁盘加载新的搜索器.因此，它不能让它的任何内部缓存升温等等，并且通常承受着巨大的压力.
After making those changes, I'd see if your reads are still slow -- I think it's quite possible that the cause of that is that every time you search, your write/commit to Solr is causing it to have to load up a fresh searcher from disk. So, it can't allow any of its internal caches to warm, etc., and is generally under a tremendous amount of strain.
希望有帮助！
这篇关于一旦我的应用程序爬升到 >，Sunspot-Solr 就会放慢速度1000 个对象 [包括 Solr 日志]的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
