核心数据使用setFetchLimit在块中的fetchrequest上进行迭代，仅处理一半记录 [英] Core data iterate over fetchrequest in chunks with setFetchLimit only processing half the records

问题描述

我正在尝试以一定大小(batchSize)的块处理许多对象.该循环似乎有效，但仅处理一半的记录.相关的代码段是:

I am trying to process a lot of objects in chunks of a certain size (batchSize). This loop seems to work, but it processes only half the records. Relevant piece of code is:

{
//Prepare fetching products without images in the database
NSFetchRequest * productFetchRequest = [NSFetchRequest fetchRequestWithEntityName:@"Product"];

//Sort by last changed photo first
NSSortDescriptor *sortDescriptor = [[NSSortDescriptor alloc] initWithKey:@"photoModificationDate" ascending:NO];
[productFetchRequest setSortDescriptors:@[sortDescriptor]];

NSPredicate *predicate = [NSPredicate predicateWithFormat: predicateString];
[productFetchRequest setPredicate:predicate];

//First get the total count
NSUInteger numberOfProducts = [self.backgroundMOC countForFetchRequest: productFetchRequest error: &error];
NSLog(@"Getting images for: %d products", numberOfProducts);

//Then set the batchsize to get chunks of data
NSUInteger batchSize = 25;
[productFetchRequest setFetchBatchSize: batchSize];
[productFetchRequest setFetchLimit:batchSize];

//Fetch the products in batches
for (NSUInteger offset = 0; offset < numberOfProducts; offset += batchSize) {
    @autoreleasepool {
        [productFetchRequest setFetchOffset: offset];
        NSArray * products = [self.backgroundMOC executeFetchRequest:productFetchRequest error:&error];
        NSLog(@"Offset: %d, number of products: %d", offset, [products count]);
        if (!products) {
            return NO;
        }

        for (Product * product in products) {
            NSLog(@"Downloading photo for product: %@", product.number);
            [self downLoadAndStoreImageForProduct:product];
        }
        [self saveAndResetBackgroundMOC];
    }
}

return YES;

}

该日志显示，对于计数的前一半(numberOfProducts)，它按预期方式工作.因此，处理了25种产品的大块.在上半年之后，循环中的fetchrequest结果为0条记录. 如果我再次重试相同的代码，则只再次处理(剩余)记录的一半，因此总数为3/4. 我究竟做错了什么? 请注意，managedObjectContext不仅会保存，而且在保存后还会重置以节省内存.如果我不这样做，则在下载约3000张图片后，程序始终崩溃.

The log shows that for the first half of the count (numberOfProducts), it works as expected. So chunks of 25 products are processed. After that first half, the fetchrequest in the loop has 0 records as a result. If I retry the same code again, again only half of the (remaining) records is processed, so 3/4 in total. What am I doing wrong? Note that the managedObjectContext is not only saved, but also reset after the save to save memory. If I do not do this in chunks, the program crashes consistantly after downloading about 3000 pictures.

推荐答案

问题出在谓词中.它获取所有没有图像的产品.如果我下载图像，则谓词的结果集会在后续提取时更改，并且每次都会变小.解决方案是按相反顺序处理结果集.因此更改:

The problem is in the predicate. It fetches all products without an image. If I download images, the result set for the predicate changes on a subsequent fetch and gets smaller every time. The solution is to process the result set in reverse order. So change:

for (NSUInteger offset = 0; offset < numberOfProducts; offset += batchSize)

进入:

for (NSInteger offset = MAX(numberOfProducts - batchSize, 0); offset > 0; offset -= batchSize)

这篇关于核心数据使用setFetchLimit在块中的fetchrequest上进行迭代，仅处理一半记录的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！