我将如何实现“尾巴"?通过 HTTP 使用 Python Tornado? [英] How would I implement "tail" over HTTP with Python Tornado?
问题描述
我正在尝试使用 Python 通过 HTTP 实现诸如tail -f"之类的东西.目前，我正在尝试使用 Tornado，但它一次只处理一个连接，即使我执行异步请求也是如此.
导入套接字导入子流程导入 tornado.gen 作为 gen导入 tornado.httpserver导入 tornado.ioloop导入 tornado.iostream导入 tornado.options导入 tornado.web从 tornado.options 导入定义，选项定义(端口"，默认值=8888，帮助=在给定端口上运行"，类型=整数)定义(输入文件"，默认=test.txt"，help="我们将'tail'的文件的路径",类型=str)类 MainHandler(tornado.web.RequestHandler):@tornado.web.asynchronous@gen.engine定义获取(自我):打印得到请求"输入文件 = 打开(选项.输入文件)p = subprocess.Popen("./nettail.py",标准输入=输入文件，标准输出=子进程.PIPE)port_number = int(p.stdout.readline().strip())self.write("")self.write("你好，世界\n")self.flush()s = socket.socket(socket.AF_INET, socket.SOCK_STREAM, 0)流 = tornado.iostream.IOStream(s)产量 gen.Task(stream.connect, ("127.0.0.1", port_number))为真:数据 = 产量 gen.Task(stream.read_until, "\n")self.write(数据)self.flush()定义主():tornado.options.parse_command_line()application = tornado.web.Application([(r"/", MainHandler),])http_server = tornado.httpserver.HTTPServer(application)http_server.listen(options.port)tornado.ioloop.IOLoop.instance().start()如果 __name__ == "__main__":主要的()我开始的过程是一个简单的尾巴"，它输出到一个套接字.
随机导入进口插座导入系统导入时间#创建一个 INET、流媒体套接字s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)# 打开连接.尝试:对于 xrange(5) 中的尝试编号:port_number = random.randint(9000, 65000)尝试:s.bind(("localhost", port_number))除了socket.error:继续# 我们绑定成功了！sys.stdout.write("{0}".format(port_number))sys.stdout.write("\n")sys.stdout.flush()休息#成为服务器套接字s.听(5)# 接受一个连接.尝试:(clientsocket, 地址) = s.accept()为真:行 = sys.stdin.readline()如果不是行:时间.睡眠(1)继续clientsocket.sendall(行)最后:clientsocket.close()最后:s.close()./nettail.py 工作正常，但 Tornado HTTP 服务器一次只处理一个请求.
我想使用长时间运行的持久 HTTP 连接来执行此操作，因为它与旧浏览器兼容.我知道 Web Sockets 是在现代浏览器中实现的方式.
我在 Linux 和 Solaris 上运行它，而不是 Windows.这意味着我可以在文件上使用 tornado.iostream，而不是通过套接字程序.尽管如此，这不是一个记录在案的特性，所以我为每个连接启动了一个套接字程序.
经过多次调试，发现这个尾服务器并没有阻塞.
我试图在打开两个 Firefox 窗口的情况下测试并发连接，但 Firefox 在第一个窗口被手动停止之前不会开始获取第二个窗口.我猜 Firefox 不喜欢有两个并发的 HTTP 连接来获取相同的资源.
打开 Firefox 窗口和 Chromium 窗口，我可以看到推送到两个选项卡的tail"输出.
谢谢大家的帮助.@abernert 的评论特别有帮助.
在即将发布的 Tornado 2.4.2 版本中，一个 "Pipe" IOStream 已实现.使用这个和常规的尾巴"大大简化了代码.
导入子流程导入 tornado.httpserver导入 tornado.ioloop导入 tornado.iostream导入 tornado.options导入 tornado.web从 tornado.options 导入定义，选项定义(端口"，默认值=8888，帮助=在给定端口上运行"，类型=整数)定义(输入文件"，默认=test.txt"，help="我们将'tail'的文件的路径",类型=str)类 MainHandler(tornado.web.RequestHandler):@tornado.web.asynchronous定义获取(自我):打印得到请求"self.p = subprocess.Popen(["tail", "-f", options.inputfile, "-n+1"],标准输出=子进程.PIPE)self.write("")self.write("你好，世界\n")self.flush()self.stream = tornado.iostream.PipeIOStream(self.p.stdout.fileno())self.stream.read_until("\n", self.line_from_nettail)def on_connection_close(self, *args, **kwargs):"""连接关闭时清理nettail进程."""打印连接已关闭！！！！"self.p.terminate()tornado.web.RequestHandler.on_connection_close(self, *args, **kwargs)def line_from_nettail(self, data):self.write(数据)self.flush()self.stream.read_until("\n", self.line_from_nettail)定义主():tornado.options.parse_command_line()application = tornado.web.Application([(r"/", MainHandler),])http_server = tornado.httpserver.HTTPServer(application)http_server.listen(options.port)tornado.ioloop.IOLoop.instance().start()如果 __name__ == "__main__":主要的()I'm trying to implement something like "tail -f" over HTTP with Python. Currently, I'm trying to use Tornado, but it only is handling one connection at a time, even when I do asynchronous requests.
import socket
import subprocess
import tornado.gen as gen
import tornado.httpserver
import tornado.ioloop
import tornado.iostream
import tornado.options
import tornado.web
from tornado.options import define, options
define("port", default=8888, help="run on the given port", type=int)
define(
"inputfile",
default="test.txt",
help="the path to the file which we will 'tail'",
type=str)
class MainHandler(tornado.web.RequestHandler):
@tornado.web.asynchronous
@gen.engine
def get(self):
print "GOT REQUEST"
inputfile = open(options.inputfile)
p = subprocess.Popen(
"./nettail.py",
stdin=inputfile,
stdout=subprocess.PIPE)
port_number = int(p.stdout.readline().strip())
self.write("<pre>")
self.write("Hello, world\n")
self.flush()
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM, 0)
stream = tornado.iostream.IOStream(s)
yield gen.Task(stream.connect, ("127.0.0.1", port_number))
while True:
data = yield gen.Task(stream.read_until, "\n")
self.write(data)
self.flush()
def main():
tornado.options.parse_command_line()
application = tornado.web.Application([
(r"/", MainHandler),
])
http_server = tornado.httpserver.HTTPServer(application)
http_server.listen(options.port)
tornado.ioloop.IOLoop.instance().start()
if __name__ == "__main__":
main()
The process I am starting is a simple "tail" which outputs to a socket.
import random
import socket
import sys
import time
#create an INET, STREAMing socket
s = socket.socket(
socket.AF_INET, socket.SOCK_STREAM)
# Open the connection.
try:
for attempt_number in xrange(5):
port_number = random.randint(9000, 65000)
try:
s.bind(("localhost", port_number))
except socket.error:
continue
# We successfully bound!
sys.stdout.write("{0}".format(port_number))
sys.stdout.write("\n")
sys.stdout.flush()
break
#become a server socket
s.listen(5)
# Accept a connection.
try:
(clientsocket, address) = s.accept()
while True:
line = sys.stdin.readline()
if not line:
time.sleep(1)
continue
clientsocket.sendall(line)
finally:
clientsocket.close()
finally:
s.close()
./nettail.py works as I expect, but the Tornado HTTP server is only handling one request at a time.
I would like to use long-running, persistent HTTP connections to do this, as it is compatible with older browsers. I understand that Web Sockets is how it would be done in modern browsers.
Edit: I'm running this on Linux and Solaris, not Windows. That means I could use tornado.iostream on the file, rather than through a sockets program. Still, that is not a documented feature, so I launch a sockets program for each connection.
After doing some more debugging, it turns out that this tail server was not blocking, after all.
I was trying to test concurrent connections with two windows of Firefox open, but Firefox would not start fetching the second window until the first window was manually stopped. I guess Firefox does not like to have two concurrent HTTP connections to fetch the same resource.
Opening a Firefox window and a Chromium window, I can see the "tail" output pushed to both tabs.
Thank you for all your help. @abarnert's comments were especially helpful.
Edit:
In the to-be-release 2.4.2 version of Tornado, a "Pipe" IOStream is implemented. Using this and regular "tail" simplified the code a lot.
import subprocess
import tornado.httpserver
import tornado.ioloop
import tornado.iostream
import tornado.options
import tornado.web
from tornado.options import define, options
define("port", default=8888, help="run on the given port", type=int)
define(
"inputfile",
default="test.txt",
help="the path to the file which we will 'tail'",
type=str)
class MainHandler(tornado.web.RequestHandler):
@tornado.web.asynchronous
def get(self):
print "GOT REQUEST"
self.p = subprocess.Popen(
["tail", "-f", options.inputfile, "-n+1"],
stdout=subprocess.PIPE)
self.write("<pre>")
self.write("Hello, world\n")
self.flush()
self.stream = tornado.iostream.PipeIOStream(self.p.stdout.fileno())
self.stream.read_until("\n", self.line_from_nettail)
def on_connection_close(self, *args, **kwargs):
"""Clean up the nettail process when the connection is closed.
"""
print "CONNECTION CLOSED!!!!"
self.p.terminate()
tornado.web.RequestHandler.on_connection_close(self, *args, **kwargs)
def line_from_nettail(self, data):
self.write(data)
self.flush()
self.stream.read_until("\n", self.line_from_nettail)
def main():
tornado.options.parse_command_line()
application = tornado.web.Application([
(r"/", MainHandler),
])
http_server = tornado.httpserver.HTTPServer(application)
http_server.listen(options.port)
tornado.ioloop.IOLoop.instance().start()
if __name__ == "__main__":
main()
这篇关于我将如何实现“尾巴"?通过 HTTP 使用 Python Tornado?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
