Python-如何使此代码异步? [英] Python - How can I make this code asynchronous?

问题描述

下面的代码说明了我的问题:

Here's some code that illustrates my problem:

def blocking1():
    while True:
        yield 'first blocking function example'

def blocking2():
    while True:
        yield 'second blocking function example'

for i in blocking1():
    print 'this will be shown'

for i in blocking2():
    print 'this will not be shown'

我有两个包含while True循环的函数.这些将产生数据，然后我将其记录到某个地方(最有可能记录到sqlite数据库).

I have two functions which contain while True loops. These will yield data which I will then log somewhere (most likely, to an sqlite database).

我一直在研究线程，并使其正常工作.但是，我不是很喜欢...我想做的是使我的阻塞函数异步.像这样:

I've been playing around with threading and have gotten it working. However, I don't really like it... What I would like to do is make my blocking functions asynchronous. Something like:

def blocking1(callback):
    while True:
        callback('first blocking function example')

def blocking2(callback):
    while True:
        callback('second blocking function example')

def log(data):
    print data

blocking1(log)
blocking2(log)

如何在Python中实现呢?我已经看到asyncore附带了标准库，该游戏中的大名是Twisted，但是这两个似乎都用于套接字IO.

How can I achieve this in Python? I've seen the standard library comes with asyncore and the big name in this game is Twisted but both of these seem to be used for socket IO.

如何异步与套接字无关的阻止功能?

How can I async my non-socket related, blocking functions?

推荐答案

阻塞函数是不会返回但仍使您的进程处于空闲状态的函数-无法完成更多工作.

A blocking function is a function which doesn't return, but still leaves your process idle - unable to complete more work.

您要我们将您的阻止功能设为非阻止.但是，除非您正在编写操作系统，否则您没有任何阻止功能.您可能因为调用了阻塞的系统调用而具有阻塞的功能，或者可能因为进行了大量的计算而使它们阻塞"了.

You're asking us to make your blocking functions non-blocking. However – unless you're writing an operating system – you don't have any blocking functions. You might have functions which block because they make calls to blocking system calls, or you might have functions which "block" because they do a lot of computation.

如果不使基础系统调用成为非阻塞，则使前一种类型的函数成为非阻塞是不可能的.取决于该系统调用是什么，如果不向程序中添加事件循环，可能很难使其无阻塞.您不仅需要拨打电话并且不阻塞电话，还必须再次拨打电话以确定该电话的结果将传递到您可以关联的位置.

Making the former type of function non-blocking is impossible without making the underlying system call non-blocking. Depending on what that system call is, it may be difficult to make it non-blocking without also adding an event loop to your program; you don't just need to make the call and have it not block, you also have to make another call to determine that the result of that call will be delivered somewhere you could associate it.

这个问题的答案是一个很长的python程序，并且对不同的OS接口以及它们如何工作有很多解释，但是幸运的是，我已经在另一个站点上写了这个答案.我称它为扭曲.如果您的特定任务已经由扭曲反应堆支持 ，那么您很幸运.否则，只要您的任务映射到某些现有的操作系​​统概念，就可以扩展反应堆来支持它.实际上，这些机制中只有两种:曾经存在的每个明智的操作系统上的文件描述符，以及Windows上的I/O完成端口.

The answer to this question is a very long python program and a lot of explanations of different OS interfaces and how they work, but luckily I already wrote that answer on a different site; I called it Twisted. If your particular task is already supported by a Twisted reactor, then you're in luck. Otherwise, as long as your task maps to some existing operating system concept, you can extend a reactor to support it. Practically speaking there are only 2 of these mechanisms: file descriptors on every sensible operating system ever, and I/O Completion Ports on Windows.

在另一种情况下，如果您的函数消耗大量CPU，因此不返回，则它们实际上并没有阻塞；您的过程仍然很艰辛，正在完成工作.有三种解决方法:

In the other case, if your functions are consuming a lot of CPU, and therefore not returning, they're not really blocking; your process is still chugging along and getting work done. There are three ways to deal with that:

• 单独的线程
• 单独的过程
• 如果您有一个事件循环，请编写一个定期产生的任务，方法是将其编写为可以做一些工作的方式，然后要求事件循环在不久的将来恢复它，以便允许其他任务执行运行.

在Twisted中，可以用多种方法来完成这最后一种技术，但这是一个语法上很方便的技巧，使之变得容易:

In Twisted this last technique can be accomplished in various ways, but here's a syntactically convenient trick that makes it easy:

from twisted.internet import reactor
from twisted.internet.task import deferLater
from twisted.internet.defer import inlineCallbacks, returnValue

@inlineCallbacks
def slowButSteady():
    result = SomeResult()
    for something in somethingElse:
        result.workHardForAMoment(something)
        yield deferLater(reactor, 0, lambda : None)
    returnValue(result)

这篇关于Python-如何使此代码异步?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！