通过python的多部分/线程下载器？ [英] A multi-part/threaded downloader via python?

问题描述

我已经在线查看了一些线程下载器，甚至一些多部分下载器（HTTP）。

I've seen a few threaded downloaders online, and even a few multi-part downloaders (HTTP).

我没有一起看过他们的类/功能。

I haven't seen them together as a class/function.

如果你们中有任何一个类/功能躺在那里，那我可以下载到我需要抓取多个文件的任何应用程序，我将非常有责任。

If any of you have a class/function lying around, that I can just drop into any of my applications where I need to grab multiple files, I'd be much obliged.

如果有库/框架（或程序的后台）结束）这样做，请指引我吗？

If there is there a library/framework (or a program's back-end) that does this, please direct me towards it?

推荐答案

Threadpool 通过Christopher Arndt可能是您正在寻找的。我使用这个易于使用的面向对象的线程池框架为了你描述的确切目的，它的作品很棒。请参阅链接页面底部的使用示例。它真的很容易使用：只需定义三个函数（其中一个是可选的异常处理程序代替默认处理程序），并且您正在进行中。

Threadpool by Christopher Arndt may be what you're looking for. I've used this "easy to use object-oriented thread pool framework" for the exact purpose you describe and it works great. See the usage examples at the bottom on the linked page. And it really is easy to use: just define three functions (one of which is an optional exception handler in place of the default handler) and you are on your way.

来自 http：//www.chrisarndt .de / projects / threadpool / ：

• 面向对象，可重用设计


• 提供回调机制来处理从工作线程返回的结果。


• WorkRequest对象包含分配给工作线程的任务，并允许轻松将任意数据传递给回调。 / li>
  
• 使用Queue类解决了大多数锁定问题。


• 所有工作线程都是守护进程，所以在主程序退出时退出，不需要加入。


• 线程一旦创建就开始运行。无需启动或停止它们。您可以随时增加或减少池大小，多余的线程将在完成当前任务后立即退出。


• 在您拥有线程后，您不需要保留对线程的引用分配最后一个任务给它。你只是说：不要回来找工作，当你完成了！


• 线程不要在等待被分配一个任务时吃掉周期，他们只是当任务队列为空时（尽管每隔几秒醒来检查是否被关闭）阻止。

• Object-oriented, reusable design
• Provides callback mechanism to process results as they are returned from the worker threads.
• WorkRequest objects wrap the tasks assigned to the worker threads and allow for easy passing of arbitrary data to the callbacks.
• The use of the Queue class solves most locking issues.
• All worker threads are daemonic, so they exit when the main program exits, no need for joining.
• Threads start running as soon as you create them. No need to start or stop them. You can increase or decrease the pool size at any time, superfluous threads will just exit when they finish their current task.
• You don't need to keep a reference to a thread after you have assigned the last task to it. You just tell it: "don't come back looking for work, when you're done!"
• Threads don't eat up cycles while waiting to be assigned a task, they just block when the task queue is empty (though they wake up every few seconds to check whether they are dismissed).

一个href =http://pypi.python.org/pypi/threadpool =nofollow noreferrer> http://pypi.python.org/pypi/threadpool ，easy_install或作为subversion checkout （见项目主页）。

Also available at http://pypi.python.org/pypi/threadpool, easy_install, or as a subversion checkout (see project homepage).

这篇关于通过python的多部分/线程下载器？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！