Sqlite或Core Data更新超过50000条记录 [英] Sqlite or Core Data to update more then 50000 records

问题描述

我目前正在为项目使用coredata。但是，当api返回54000个需要更新的对象时，用户必须等待近2个小时。
这是当前项目的主要问题，我正在考虑使用sqlite而不再使用coredata来更新数千个对象。

I'm currently using coredata for my project. But when the api returns 54000 objects that the app need to update, the user has to wait almost 2 hours. It's the major problem for the current project and I am thinking to use sqlite and not using coredata anymore to update thousands of objects.

这是正确的决定吗使用Sqlite还是对CoreData有任何建议？我不能决定任何帮助都会很棒。谢谢。

Is it a right decision to use Sqlite or is there any suggestion for CoreData? I can't decide. Any help will be great. Thank you.

这是我在做什么：

NSManagedObjectContext *privateObjectContext = [AppDelegate appDelegate].privateManagedObjectContext;
    [privateObjectContext performBlock:^{

      int i = 1;
      for (NSDictionary *item in itemlist) {
            i++;

            [fetchRequest setPredicate:[NSPredicate predicateWithFormat:
                                        @"itemID == %@",[item objectForKey:@"item_id"]
                                        ]];
            NSError *error;
            NSMutableArray *inventories = [[NSMutableArray alloc]initWithArray:
                                           [privateObjectContext executeFetchRequest:fetchRequest
                                                                               error:&error]];
            ItemManagedObject *itemMO;

            if(inventories.count){
                itemMO = inventories.firstObject;
            }else{
                itemMO = [NSEntityDescription insertNewObjectForEntityForName:@"ItemObject"
                                                       inManagedObjectContext:privateObjectContext];
            }
            [itemMO prepareWithDictionary:item];
        }

        NSError *error;
        if (![privateObjectContext save:&error]) {
            completionHandler(NO);
        }
}

推荐答案

2时间很长。

但是，您可以通过减少核心数据的工作量来完善代码。

Yet you can massage your code by having core data do less work. Much less work.

1. 执行单个提取请求，而不是54K提取请求


2. 当属性值不变时不要调用托管对象属性设置器，这样就不会不必要地将任何对象标记为脏对象，并且Core Data不必执行昂贵但无用的操作

1. Perform a single fetch request instead of 54K fetch requests
2. Don't call a managed object property setter when a property value does not change, so that no object is unnecessarily flagged as dirty, and Core Data does not have to perform a costly but useless update of the object when the "save" method is invoked.

这将大大减少Core Data执行的工作量，并且

This will dramatically reduce the amount of work performed by Core Data, and the performance of your application.

第二点很简单，但很冗长：在调用设置方法之前，将每个属性值与字典值进行比较。

The second point is easy, but very verbose: compare each individual property values with dictionary values before calling setters.

第一点需要更改算法：

执行单个提取请求，并按ID排序（使用[NSFetchRequest setSortDescriptors：]）

Perform a single fetch request, sorted by id (with [NSFetchRequest setSortDescriptors:])

按ID对字典排序（使用[NSArray sortedArray ...]）

Sort dictionaries by id (with [NSArray sortedArray...])

同步两个排序列表（它i对两个列表都进行排序的最重要的）：

Synchronize the two sorted lists (it is paramount that both lists are sorted):

NSEnumerator *itemMOEnum = [itemMOs objectEnumerator];
NSEnumerator *dicEnum = [dictionaries objectEnumerator];
ItemManagedObject *itemMO = [itemMOEnum nextObject];
NSDictionary *itemDic = [dicEnum nextObject];

while (itemDic) {
    NSComparisonResult comparison = itemMO ? [itemDic[@"item_id"] compare:itemMO.itemID] : NSOrderedAscending;
    switch (comparison) {
        case NSOrderedSame:
            // id present in both lists: update
            [itemMO prepareWithDictionary:itemDic];

            itemMO = [itemMOEnum nextObject];
            itemDic = [dicEnum nextObject];
            break;

        case NSOrderedAscending: {
            // id present only in dictionaries: create
            itemMO = [NSEntityDescription insertNewObjectForEntityForName:@"ItemObject"
                                          inManagedObjectContext:privateObjectContext];
            [itemMO prepareWithDictionary:itemDic];

            itemDic = [dicEnum nextObject];
        } break;

        case NSOrderedDescending:
            // id present only in managed object: delete or do nothing
            itemMO = [itemMOEnum nextObject];
            break;
    }
}

while (itemMO) {
    // id present only in managed object: delete or do nothing
    itemMO = [itemMOEnum nextObject];
}

并保存。

最后，也许SQLite会更快（请参见 https://github.com/groue/ GRDB.swift / wiki / Performance ，以尝试比较Core Data和SQLite库的性能。

Finally, maybe SQLite will be faster (see https://github.com/groue/GRDB.swift/wiki/Performance for an attempt at comparing the performance of Core Data with SQLite libraries).

但是SQLite不会变将慢速算法转换为快速算法。

这篇关于Sqlite或Core Data更新超过50000条记录的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！