了解异步/多处理在Python [英] Understanding Asynchronous/Multiprocessing in Python

问题描述

可以说我有一个函数：

from time import sleep

def doSomethingThatTakesALongTime(number):
  print number
  sleep(10)

然后我把它称为一个for循环

and then I call it in a for loop

for number in range(10):
  doSomethingThatTakesALongTime(number)

我

如何进行此项设置，使其只需要10秒TOTAL打印出来：

How can I set this up so that it only takes 10 seconds TOTAL to print out:

$ 0123456789

而不是采取100秒。如果有帮助，我将使用您提供做异步Web刮的信息。即我有我想要访问的网站的名单，但我想同时看望他们，而不是等待每一个来完成。

Instead of taking 100 seconds. If it helps, I'm going to use the information YOU provide to do asynchronous web scraping. i.e. I have a list of sites I want to visit, but I want to visit them simultaneously, rather than wait for each one to complete.

推荐答案

看看scrapy框架。它是专门针对网络刮是非常好的。这是asynchronus和建立在扭曲的框架。

Take a look at scrapy framework. It's intended specially for web scraping and is very good. It is asynchronus and built on twisted framework.

http://scrapy.org/

这篇关于了解异步/多处理在Python的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！