多个urllib2连接 [英] Multiple urllib2 connections

问题描述

我想同时下载多张图像.为此，我使用线程，每个线程都使用urllib2模块下载图像.我的问题是，即使在线程(几乎同时)启动的情况下，也像在单线程环境中一样，一张一张地下载图像.

I want to download multiple images at the same time. For that I'm using threads, each one downloading an image, using urllib2 module. My problem is that even if threads starts (almost) simultaneously, the images are downloaded one by one, like in a single-threaded environment.

这是线程函数:

def updateIcon(self, iter, imageurl):
    req = urllib2.Request('http://site.com/' + imageurl)
    response = urllib2.urlopen(req)
    imgdata = response.read()
    gobject.idle_add(self.setIcon, iter, imgdata)

调试我的代码后，我发现下载似乎停滞在"response = urllib2.urlopen(req)"行中.有什么问题?这是因为线程模块还是urllib2?我该如何解决?

Debugging my code I found that downloads seems to get stuck at "response = urllib2.urlopen(req)" line. What's the problem? It's because the threading module or urllib2? How I can fix that?

提前谢谢

推荐答案

考虑使用 urllib3 .它支持连接池和通过进程(不是线程)的多个并发请求.它应该解决这个问题.如果您联系许多不同的站点，请小心垃圾收集连接池，因为每个站点都有自己的池.

Consider using urllib3. It supports connection pooling and multiple concurrent requests via processes (not threads). It should solve this problem. Be careful to garbage collect connection pools if you contact many different sites, since each site gets its own pool.

这篇关于多个urllib2连接的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！