Python一次访问多个网页 [英] Python accessing multiple webpages at once

问题描述

我有一个tkinter GUI，一次从多个网站下载数据。我为每次下载运行一个单独的线程（约28）。一个GUI进程的线程是否太多？因为它真的很慢，每个单独的页面大约需要1到2秒，但是当它们全部运行一次时，它需要40秒以上。有什么办法可以缩短下载所有页面所需的时间吗？谢谢，谢谢。

解决方案

这可能是GIL（全局解释器锁）。 Python在许多线程中有一些性能问题。

您可以尝试twisted.web.getPage（请参阅 http://twistedmatrix.com/projects/core/documentation/howto/async.html 一点下来）。
我没有基准。
但是在该页面上的示例中，添加28个延迟来查看它的速度是多快可以给你一个可比的结果。
请记住，您必须使用gtk反应堆，并加入编曲风格。

I have a tkinter GUI that downloads data from multiple websites at once. I run a seperate thread for each download (about 28). Is that too much threads for one GUI process? because it's really slow, each individual page should take about 1 to 2 seconds but when all are run at once it takes over 40 seconds. Is there any way I can shorten the time it takes to download all the pages? Any help is appreciated, thanks.

解决方案

It's probably the GIL (global interpreter lock) that gets in your way. Python has some performance problems with many threads.

You could try twisted.web.getPage (see http://twistedmatrix.com/projects/core/documentation/howto/async.html a bit down the page). I don't have benchmarks for that. But taking the example on that page and adding 28 deferreds to see how fast it is will give you a comparable result pretty fast. Keep in mind, that you'd have to use the gtk reactor and get into twisteds programming style, though.

这篇关于Python一次访问多个网页的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！