AFNetworking + NsOperationQueue - 下载数千张图片 [英] AFNetworking + NsOperationQueue - Download thousands of Images

问题描述

我正在处理一项任务（仅限iOS5 +），涉及从服务器下载数千张图像。图像属于某些类别，每个类别可以有数百个图像。我需要做的是： -

1）如果应用程序处于活动状态，请确保应用程序在后台下载任何丢失的图像（即使用户正在浏览其他应用程序）应用程序中与照片无关的区域）。

2）当用户点击照片类别时，必须将该类别中的图像作为高优先级下载，因为那些是需要立即可见的图像。 / p>

所有上述情况仅在图像尚未离线时才会发生。下载后，图像将从本地存储中使用。

要解决此问题，我使用的逻辑是： -

1）在AppDelegate.m中，在 applicationDidBecomeActive 中，我开始下载任何丢失的图像。为此，我进行了核心数据查询，找出缺少的图像，并开始在具有BACKGROUND优先级的线程中下载它们。这样的事情： -

  dispatch_queue_t imageDownloadQueue = dispatch_get_global_queue（DISPATCH_QUEUE_PRIORITY_BACKGROUND，0）; 
 dispatch_async（imageDownloadQueue，^ {
 [DataDownloader downloadMissingImages]; 
}）; 
 dispatch_release（imageDownloadQueue）; 
  

downloadMissingImages 代码如下所示： -

  NSOperationQueue * downloadQueue = [[NSOperationQueue alloc] init]; 
 downloadQueue.maxConcurrentOperationCount = 20; 
 
 for（MyImage * img in matches）
 {
 NSURLRequest * request = [NSURLRequest requestWithURL：img.photoUrl]; 
 AFImageRequestOperation * operation = [AFImageRequestOperation imageRequestOperationWithRequest：请求成功：^（UIImage * image）{
 
 [MyImage imageFromAPI：image inManagedObjectContext：document.managedObjectContext]; 
 
 NSLog（@成功下载％@的图像，img.title）; 
}]; 
 
 [downloadQueue addOperation：operation]; 
} 
  

这有效，但它会阻止主用户界面，应用程序会在一段时间后崩溃。这是我尝试下载大约700张图片的时候。随着更多的图像，它肯定会崩溃。

2）当用户点击某个类别时，我需要先下载这些图像，因为它们必须立即显示给用户。我不知道如何中断missImages调用并告诉它开始在其他人之前下载某些图像。

因此，基本上，我需要在后台下载所有丢失的图像，但如果用户正在浏览照片类别，则这些图像必须在下载队列中具有高优先级。

我不知道如何有效地运作。有什么想法？

崩溃日志看起来像这样

  PAPP（ 36373,0xb065f000）malloc：*** mmap（size = 16777216）失败（错误代码= 12）
 ***错误：无法分配区域
 ***在malloc_error_break中设置断点以进行调试
 PAPP（36373,0xb065f000）malloc：*** mmap（size = 16777216）失败（错误代码= 12）
 ***错误：无法分配区域
 *** set malloc_error_break中的断点调试
 Jun 24 11:39:45 MacBook-Pro.local PAPP [36373]<错误>：ImageIO：JPEG内存不足（案例4）
  

提前致谢。

解决方案

关于崩溃，我猜你的应用程序因两种选择中的任何一种而被杀死：

1. 应用程序无响应（因此没有响应到iOS的哨兵过程）;




2. 在循环中使用的内存太多，创建了超过700个请求操作。

澄清什么实际上，您应该提供有关崩溃的更多信息（控制台日志）。在任何情况下，修复程序都会以每个10或20个块的形式加载图像（如果你愿意的话，你甚至可以逐个加1，我没有看到太多问题）。

关于第二点，这是怎么回事：

1. 下载更高优先级的图像线程（通过异步下载，当然，以避免阻止UI）;




2. 在开始下载离线图像之前，检查图像是否有已经通过更高优先级下载同时下载了。

为了更好地处理第2点，你会可能需要排队您的自定义操作而不是 AFImageRequestOperation ，以便在实际下载之前进行检查。

编辑：

关于以块的形式下载图像，您可以做的是使用调度组对您的网络操作进行分组：

  dispatch_group_t group = dispatch_group_create（）; 
 
< your_core_data_query> （b）
 
 
 dispatch_group_enter（group）; 
 dispatch_group_enter（group）; 
 
 NSURLRequest * request = [NSURLRequest requestWithURL：img.photoUrl]; 
 AFImageRequestOperation * operation = [AFImageRequestOperation imageRequestOperationWithRequest：请求成功：^（UIImage * image）{
 [MyImage imageFromAPI：image inManagedObjectContext：document.managedObjectContext]; 
 NSLog（@成功下载％@的图像，img.title）; 
 
 dispatch_group_leave（group）; //< ==注意这个
}]; 
 
} 
 
 dispatch_group_wait（group，DISPATCH_TIME_FOREVER）; 
 dispatch_release（group）; 
  

在此示例中，我使用调度组对多个异步操作进行分组并等待它们被执行全部;当 dispatch_group_wait 返回时，你可以执行另一轮（查询核心数据然后调度操作）。

关于你的其他问题（如何检查更高优先级队列是否已下载某个图像），您应该在执行每个 AFImageRequestOperation 之前执行核心数据查询;一种可能性是派生你自己的类并覆盖 start 方法来进行检查。

在两个帐户上，你可以通过一次下载图像来简化所有这些的逻辑（即， for（...）循环不会出现;你只需要查询下载和下载它的下一个图像;在下载之前你检查它是否已经存在。

我建议你走这条更容易的道路。

希望有所帮助。

I am working on a task (iOS5 + only) that involves downloading thousands of images from the server. The images belong to certain categories and each category can have hundreds of images. What I need to do is this :-

1) Make sure the app downloads any missing images in background if the app is active (even when the user is browsing some other areas of the app that are not related to photos).

2) When the user clicks on a Photo Category, the images in that Category must be downloaded as high priority because those are the ones that need to be visible immediately.

All of the above happens only if the image is not already available offline. Once it's downloaded, the image will be used from local storage.

To solve this, the logic I am using is this :-

1) In AppDelegate.m, in applicationDidBecomeActive, I start downlading any missing images. To do this, I make a Core Data query, find out which images are missing, and start downloading them in a thread with BACKGROUND priority. Something like this :-

 dispatch_queue_t imageDownloadQueue = dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_BACKGROUND, 0);
dispatch_async(imageDownloadQueue, ^{
    [DataDownloader downloadMissingImages];
});
dispatch_release(imageDownloadQueue);

The downloadMissingImages code looks like this :-

NSOperationQueue *downloadQueue = [[NSOperationQueue alloc] init];
        downloadQueue.maxConcurrentOperationCount = 20;

        for(MyImage *img in matches)
        {
            NSURLRequest *request = [NSURLRequest requestWithURL:img.photoUrl];
            AFImageRequestOperation *operation = [AFImageRequestOperation imageRequestOperationWithRequest:request success:^(UIImage *image) {

                [MyImage imageFromAPI:image inManagedObjectContext:document.managedObjectContext];

                NSLog(@"Successfully downloaded image for %@", img.title);      
            }];

            [downloadQueue addOperation:operation];
        }

This works, but it blocks the main UI and the app crashes after a while. This is when I try to download about 700 images. With more images, it would certainly crash.

2) When a user clicks on a category, I need to download those images first as they must be shown to the user immediately. I am not sure how I can interrupt the missingImages call and tell it to start downloading certain images before others.

So, basically, I need to download all the missing images in the background but if the user is browsing photo category, those images must take high priority in the download queue.

I am at a loss how to get this working efficiently. Any thoughts?

The crash logs look like this

PAPP(36373,0xb065f000) malloc: *** mmap(size=16777216) failed (error code=12)
*** error: can't allocate region
*** set a breakpoint in malloc_error_break to debug
PAPP(36373,0xb065f000) malloc: *** mmap(size=16777216) failed (error code=12)
*** error: can't allocate region
*** set a breakpoint in malloc_error_break to debug
Jun 24 11:39:45 MacBook-Pro.local PAPP[36373] <Error>: ImageIO: JPEG    Insufficient memory (case 4)

Thanks in advance.

解决方案

About the crash, I guess that your app is killed due to either of two options:

1. the app becoming unresponsive (and thus not responding to the iOS sentinel process);

2. too much memory used in the loop creating over 700 request operations.

To clarify what is really happening, you should provide more info about the crash (the console log). In any case, the fix would be loading the images in chunks of maybe 10 or 20 each (you could go even 1 by 1, if you like, I don't see much problem with that).

About the second point, what about this:

1. download a higher priority image in the main thread (via an async download, of course, to avoid blocking the UI);

2. before starting downloading an "offline" image, you check whether the image has been already downloaded in the meanwhile through a "higher-priority" download.

To handle point 2 well, you would probably need to queue your custom operation instead of an AFImageRequestOperation in order to do the check before the actual download.

EDIT:

About downloading images in chunk, what you could do is using dispatch groups to group your networking operations:

dispatch_group_t group = dispatch_group_create();

<your_core_data_query>

for (...) {
    dispatch_group_enter(group);

        NSURLRequest *request = [NSURLRequest requestWithURL:img.photoUrl];
        AFImageRequestOperation *operation = [AFImageRequestOperation imageRequestOperationWithRequest:request success:^(UIImage *image) {
            [MyImage imageFromAPI:image inManagedObjectContext:document.managedObjectContext];
            NSLog(@"Successfully downloaded image for %@", img.title);

            dispatch_group_leave(group);     //<== NOTICE THIS
        }];

}

dispatch_group_wait(group, DISPATCH_TIME_FOREVER);
dispatch_release(group);

In this sample, I am using a dispatch group to group a number of async operations and wait for them being executed all; when dispatch_group_wait returns, you can execute another round of that (querying core data then dispatching ops).

About your other question (how do I check whether a higher priority queue has already downloaded a certain image), you should do a core data query before executing each AFImageRequestOperation; one possibility is deriving your own class of it and overriding the start method to do the check.

On both accounts, you could simplify much the logics of all this by downloading the images one at the time (i.e., the for (...) loop would not be there; you simply query the next image to download and download it; before downloading you check it is not already there.

I would suggest going this easier path.

Hope it helps.

这篇关于AFNetworking + NsOperationQueue - 下载数千张图片的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！