Python请求非阻塞? [英] Python Requests non-blocking?

问题描述

可能的重复:
使用 Python 请求的异步请求

python 模块 Requests 是非阻塞的吗?我在文档中没有看到任何关于阻塞或非阻塞的内容.

Is the python module Requests non-blocking? I don't see anything in the docs about blocking or non-blocking.

如果是阻塞，你会推荐哪个模块?

If it is blocking, which module would you suggest?

推荐答案

像 urllib2 一样，requests 是阻塞的.

Like urllib2, requests is blocking.

但我也不建议使用其他库.

But I wouldn't suggest using another library, either.

最简单的答案是在单独的线程中运行每个请求.除非你有数百个，否则这应该没问题.(多少百多少多少取决于您的平台.在 Windows 上，限制可能是线程堆栈有多少内存；在大多数其他平台上，截止时间更早.)

The simplest answer is to run each request in a separate thread. Unless you have hundreds of them, this should be fine. (How many hundreds is too many depends on your platform. On Windows, the limit is probably how much memory you have for thread stacks; on most other platforms the cutoff comes earlier.)

如果你确实有数百个，你可以把它们放在一个线程池中.ThreadPoolExecutor 示例concurrent.futures 页面几乎正是您所需要的；只需将 urllib 调用更改为 requests 调用即可.(如果您使用的是 2.x，请使用 futures，在 PyPI 上向后移植相同的包.)缺点是你实际上并没有一次启动所有 1000 个请求，只是第一个，比如 8 个.

If you do have hundreds, you can put them in a threadpool. The ThreadPoolExecutor Example in the concurrent.futures page is almost exactly what you need; just change the urllib calls to requests calls. (If you're on 2.x, use futures, the backport of the same packages on PyPI.) The downside is that you don't actually kick off all 1000 requests at once, just the first, say, 8.

如果你有数百个，而且它们都需要并行，这听起来像是 gevent.让它对所有内容进行猴子补丁，然后编写与使用线程编写的完全相同的代码，但生成 greenlets 而不是 Threads.

If you have hundreds, and they all need to be in parallel, this sounds like a job for gevent. Have it monkeypatch everything, then write the exact same code you'd write with threads, but spawning greenlets instead of Threads.

grequests，它直接在 requests，有效地为你做 gevent + requests 包装.对于最简单的情况，这很棒.但是对于任何重要的事情，我发现阅读明确的 gevent 代码更容易.您的里程可能会有所不同.

grequests, which evolved out of the old async support directly in requests, effectively does the gevent + requests wrapping for you. And for the simplest cases, it's great. But for anything non-trivial, I find it easier to read explicit gevent code. Your mileage may vary.

当然，如果你需要做一些真正喜欢的事情，你可能想去twisted、tornado或tulip(或等待几个月让 tulip 成为 stdlib 的一部分).

Of course if you need to do something really fancy, you probably want to go to twisted, tornado, or tulip (or wait a few months for tulip to be part of the stdlib).

这篇关于Python请求非阻塞?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！