如何在HTTP请求中执行shell命令并使用Python和Flask输出流? [英] How to execute shell command and stream output with Python and Flask upon HTTP request?

问题描述

按照这篇文章，我可以将一个日志文件 tail -f 到网页:

Following this post, I am able to tail -f a log file to a webpage:

from gevent import sleep
from gevent.wsgi import WSGIServer
import flask
import subprocess

app = flask.Flask(__name__)

@app.route('/yield')
def index():
    def inner():
        proc = subprocess.Popen(
                ['tail -f ./log'],
                shell=True,
                stdout=subprocess.PIPE
                )
        for line in iter(proc.stdout.readline,''):
            sleep(0.1)
            yield line.rstrip() + '<br/>\n'

    return flask.Response(inner(), mimetype='text/html')

http_server = WSGIServer(('', 5000), app)
http_server.serve_forever()

此方法有两个问题.

1. 关闭网页后， tail -f log 进程将持续存在.在访问 http://localhost:5000/yield n次之后将有n个尾巴处理
2. 只有1个客户端一次访问 http://localhost:5000/yield

1. The tail -f log process will linger after closing the webpage. There will be n tail process after visiting http://localhost:5000/yield n time
2. There can only be 1 client accessing http://localhost:5000/yield at a single time

我的问题是，有人访问页面时，是否可以使flask执行shell命令，而客户端关闭页面时，使烧瓶终止?类似于 tail -f log 之后的 Ctrl + C .如果没有，有哪些替代方案?为什么一次只能让1个客户端访问该页面?

My question(s) is, is it possible to make flask execute a shell command when someone visit a page and terminating the command when client close the page? Like Ctrl+C after tail -f log. If not, what are the alternatives? Why was I only able to have 1 client accessing the page at a time?

注意:我正在研究启动/停止任意shell命令的一般方式，而不是特别拖尾文件

Note: I am looking into general way of starting/stoping an arbitrary shell command instead of particularly tailing a file

推荐答案

以下是一些应完成的代码.一些注意事项:

Here is some code that should do the job. Some notes:

1. 您需要检测请求何时断开连接，然后终止proc.下面的try/except代码可以做到这一点.但是，在inner()到达终点之后，Python将尝试正常关闭套接字，这将引发异常(我认为这是socket.error，根据

1. You need to detect when the request disconnects, and then terminate the proc. The try/except code below will do that. However, after inner() reaches its end, Python will try to close the socket normally, which will raise an exception (I think it's socket.error, per How to handle a broken pipe (SIGPIPE) in python?). I can't find a way to catch this exception cleanly; e.g., it doesn't work if I explicitly raise StopIteration at the end of inner(), and surround that with a try/except socket.error block. That may be a limitation of Python's exception handling. There may be something else you can do within the generator function to tell flask to abort streaming without trying to close the socket normally, but I haven't found it.

您的主线程在proc.stdout.readline()期间阻塞，而gevent.sleep()来不及提供帮助.原则上，gevent.monkey.patch_all()可以修补标准库，以便通常会阻塞线程的函数将产生对gevent的控制权(请参阅

Your main thread is blocking during proc.stdout.readline(), and gevent.sleep() comes too late to help. In principle gevent.monkey.patch_all() can patch the standard library so that functions that would normally block the thread will yield control to gevent instead (see http://www.gevent.org/gevent.monkey.html). However, that doesn't seem to patch proc.stdout.readline(). The code below uses gevent.select.select() to wait for data to become available on proc.stdout or proc.stderr before yielding the new data. This allows gevent to run other greenlets (e.g., serve other web clients) while waiting.

Web服务器似乎在缓冲发送到客户端的前几kB数据，因此，除非在./log中添加了许多新行，否则您可能无法在Web浏览器中看到任何内容.在那之后，似乎立即发送新数据.不确定如何立即发送请求的第一部分，但这对于流服务器可能是一个很常见的问题，因此应该有一个解决方案.对于自己快速终止的命令来说，这不是问题，因为一旦终止，它们的全部输出就会发送出去.

The webserver seems to buffer the first few kB of data being sent to the client, so you may not see anything in your web browser until a number of new lines have been added to ./log. After that, it seems to send new data immediately. Not sure how to get the first part of the request to be sent right away, but it's probably a pretty common problem with streaming servers, so there should be a solution. This isn't a problem with commands that terminate quickly on their own, since their full output will be sent once they terminate.

您可能还会在代码如下:

from gevent.select import select
from gevent.wsgi import WSGIServer
import flask
import subprocess

app = flask.Flask(__name__)

@app.route('/yield')
def index():
    def inner():
        proc = subprocess.Popen(
                ['tail -f ./log'],
                shell=True,
                stdout=subprocess.PIPE,
                stderr=subprocess.PIPE
                )
        # pass data until client disconnects, then terminate
        # see https://stackoverflow.com/questions/18511119/stop-processing-flask-route-if-request-aborted
        try:
            awaiting = [proc.stdout, proc.stderr]
            while awaiting:
                # wait for output on one or more pipes, or for proc to close a pipe
                ready, _, _ = select(awaiting, [], [])
                for pipe in ready:
                    line = pipe.readline()
                    if line:
                        # some output to report
                        print "sending line:", line.replace('\n', '\\n')
                        yield line.rstrip() + '<br/>\n'
                    else:
                        # EOF, pipe was closed by proc
                        awaiting.remove(pipe)
            if proc.poll() is None:
                print "process closed stdout and stderr but didn't terminate; terminating now."
                proc.terminate()

        except GeneratorExit:
            # occurs when new output is yielded to a disconnected client
            print 'client disconnected, killing process'
            proc.terminate()

        # wait for proc to finish and get return code
        ret_code = proc.wait()
        print "process return code:", ret_code

    return flask.Response(inner(), mimetype='text/html')

http_server = WSGIServer(('', 5000), app)
http_server.serve_forever()

这篇关于如何在HTTP请求中执行shell命令并使用Python和Flask输出流?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！