如何批量执行for循环? [英] How to execute a for loop in batches?

问题描述

for x in records:
   data = {}
   for y in sObjectName.describe()['fields']
         data[y['name']] = x[y['name']]
   ls.append(adapter.insert_posts(collection, data))

我想以500的批量大小执行代码ls.append(adapter.insert_post(collection，x))，其中x应该包含500个数据字典.我可以使用double for循环和一个列表创建一个包含500个数据字典的列表，然后将其插入.我可以通过以下方式做到这一点，是否有更好的方法呢? :

I want to execute the code ls.append(adapter.insert_post(collection, x)) in the batch size of 500, where x should contain 500 data dicts. I could create a list a of 500 data dicts using a double for loop and a list and then insert it. I could do that in the following way, , is there a better way to do it? :

for x in records:
    for i in xrange(0,len(records)/500):
        for j in xrange(0,500):
            l=[]
            data = {}
            for y in sObjectName.describe()['fields']:
                data[y['name']] = x[y['name']]
                #print data
            #print data
            l.append(data)
        ls.append(adapter.insert_posts(collection, data))

    for i in xrange(0,len(records)%500):
        l=[]
        data = {}
        for y in sObjectName.describe()['fields']:
            data[y['name']] = x[y['name']]
            #print data
        #print data
        l.append(data)
    ls.append(adapter.insert_posts(collection, data))

推荐答案

我使用的一般结构如下:

The general structure I use looks like this:

worklist = [...]
batchsize = 500

for i in xrange(0, len(worklist), batchsize):
    batch = worklist[i:i+batchsize] # the result might be shorter than batchsize at the end
    # do stuff with batch

请注意，我们使用xrange的step参数来大大简化批处理.

Note that we're using the step argument of xrange to simplify the batch processing considerably.

这篇关于如何批量执行for循环?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！