ActiveRecord：替代find_in_batches？ [英] ActiveRecord: Alternative to find_in_batches?

问题描述

我有一个查询，可以加载数千个对象，我想使用 find_in_batches 来驯服它：

I have a query that loads thousands of objects and I want to tame it by using find_in_batches:

Car.includes（：member）.where（：engine => 123）。find_in_batches（batch_size：500）...

根据文档，我无法自定义排序顺序： http://www.rubydoc.info/docs/rails/4.0.0/ActiveRecord/Batches:find_in_batches

According to the docs, I can't have a custom sorting order: http://www.rubydoc.info/docs/rails/4.0.0/ActiveRecord/Batches:find_in_batches

但是，我需要 created_at DESC 的自定义排序顺序。是否有另一种方法可以像在find_in_batches中那样以块的形式运行此查询，以使一次堆上没有太多对象？

However, I need a custom sort order of created_at DESC. Is there another method to run this query in chunks like it does in find_in_batches so that not so many objects live on the heap at once?

推荐答案

嗯，我一直在想办法解决这个问题（我是问这个问题的人）。理解 find_in_batches 不允许您自定义订单是有道理的，因为可以说您按 created_at DESC 和指定batch_size为500。第一个循环从1-500开始，第二个循环从501-1000开始，依此类推。如果在第二个循环发生之前有人将新记录插入表中，该怎么办？将其放在查询结果的顶部，结果将向左移动1，第二个循环将重复。

Hm I've been thinking about a solution for this (I'm the person who asked the question). It makes sense that find_in_batches doesn't allow you to have a custom order because lets say you sort by created_at DESC and specify a batch_size of 500. The first loop goes from 1-500, the second loop goes from 501-1000, etc. What if before the 2nd loop occurs, someone inserts a new record into the table? That would be put onto the top of the query results and your results would be shifted 1 to the left and your 2nd loop would have a repeat.

您可以争论一下 created_at ASC 会很安全，但是不能保证您的应用指定了created_at值。

You could argue though that created_at ASC would be safe then, but it's not guaranteed if your app specifies a created_at value.

更新：

我为这个问题写了一个宝石： https：// github .com / EdmundMai / batched_query

I wrote a gem for this problem: https://github.com/EdmundMai/batched_query

自使用以来，我的应用程序的平均内存已减少。我强烈建议任何有类似问题的人来看看！并根据需要捐款！

Since using it, the average memory of my application has HALVED. I highly suggest anyone having similar issues to check it out! And contribute if you want!

这篇关于ActiveRecord：替代find_in_batches？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！