排队多个下载，寻找生产者消费者API [英] Queueing Multiple Downloads, looking for a producer consumer API

问题描述

我有一个应用程序(一个 servlet，但这不是很重要)，它下载一组文件并解析它们以提取信息.到目前为止，我在循环中进行了这些操作:- 在 Internet 上获取新文件- 分析它.

I have an application (a servlet, but that's not very important) that downloads a set of files and parse them to extract information. Up to now, I did those operations in a loop : - fetching new file on the Internet - analyzing it.

多线程下载管理器似乎是这个问题的更好解决方案，我想以最快的方式实现它.

A multi-threaded download-manager seems a better solution for this problem and I would like to implement it in the fastest way possible.

某些下载依赖于其他下载(因此，此集合是部分订购的).

Some of the downloads are dependant from others (so, this set is partially ordered).

多线程编程很难，如果我能找到一个 API 来做到这一点，我会很高兴.我需要将一组文件(有序)放入队列中，并获取完全下载的第一组文件.

Mutli-threaded programming is hard and if I could find an API to do that I would be quite happy. I need to put a group of files (ordered) in a queue and get the first group of files that is completely downloaded.

你知道我可以用什么库来实现这一点吗?

Do you know of any library I could use to achieve that ?

问候，斯蒂芬

推荐答案

你可以这样做:

BlockingQueue<Download> queue = new BlockingQueue<Download>();
ExecutorService pool = Executors.newFixedThreadPool(5);
Download obj = new Download(queue); 
pool.execute(obj); //start download and place on queue once completed
Data data = queue.take(); //get completely downloaded item

如果每次下载的速度不同，您可能需要使用不同类型的队列.我相信 BlockingQueue 是先进先出的.

You may have to use a different kind of queue if the speed of each download is not the same. BlockingQueue is first in first out I believe.

您可能需要考虑使用 PriorityBlockingQueue，它将根据下载对象的 Comparable 方法对其进行排序.请参阅此处的 API 了解更多详情.

You may want to look into using a PriorityBlockingQueue which will order the Download objects according to their Comparable method. See the API here for more details.

希望这会有所帮助.

这篇关于排队多个下载，寻找生产者消费者API的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！