运行多个龙卷风进程 [英] run multiple tornado processess

问题描述

我已经阅读了有关如何运行 N 个 Tornado 进程的各种文章和教程，其中 N = 内核数.我的代码运行正常，在所有 16 个内核上运行，但不知何故我设法搞砸了，我需要重新审视这一点.

I've read various articles and tutorials on how to run N number of Tornado processes, where N=number of cores. My code was working, running on all 16 cores but I somehow managed to screw it up and I need fresh eyes on this.

import tornado.ioloop
import tornado.web
import tornado.httpserver

from core import settings
from core import coreService
import services

from tornado.options import define, options, parse_command_line

define("port", default=settings.SERVER_PORT, help="run on the given port", type=int)



app = tornado.web.Application([
    (r'/upload', coreService.Upload)
])

if __name__ == "__main__":
    tornado.options.parse_command_line()
    server = tornado.httpserver.HTTPServer(app, max_buffer_size=1024*1024*201)
    server.bind(options.port)
    # autodetect cpu cores and fork one process per core
    server.start(0)
    try:        
        print 'running on port %s' % options.port
        tornado.ioloop.IOLoop.instance().start()

    except KeyboardInterrupt:
        tornado.ioloop.IOLoop.instance().stop()

此代码引发此错误:

File "/opt/tornado/tornado/process.py", line 112, in fork_processes
    raise RuntimeError("Cannot run in multiple processes: IOLoop instance "
RuntimeError: Cannot run in multiple processes: IOLoop instance has already been initialized. You cannot call IOLoop.instance() before calling start_processes()

我只是没看到.谢谢

:

正如本所说，我的一种方法给我带来了麻烦.这是该方法的代码，有人可能会从中受益:

As Ben said, one of my methods was giving me trouble. This is the code of that method, someone may benefit from this:

from tornado import gen
import motor

db = motor.MotorClient().open_sync().proba
class Upload(BaseHandler):
    @gen.engine
    def post(self):
        fs = yield motor.Op(motor.MotorGridFS(db).open)

        gridin = yield motor.Op(fs.new_file)
        yield motor.Op(gridin.write, 'First part\n')
        yield motor.Op(gridin.write, 'Second part')
        yield motor.Op(gridin.close)

        print gridin._id
        self.write(str(gridin._id))
        self.finish()

编辑 2

我找到了问题的最终解决方案.正如 Ben 所指出的，上述方法给我带来了麻烦.电机文档中记录了将电机与 Tornado 应用程序结合使用的正确方法.这是一个对我有用的except:

I've found the final solution to my problem. As pointed out by Ben the method above was giving me trouble. The correct way to include Motor with Tornado application is documented on Motor documentation. Here is an except which is working for me:

if __name__ == "__main__":
    tornado.options.parse_command_line()        
    try:
        server = tornado.httpserver.HTTPServer(app, max_buffer_size=1024*1024*201)
        server.bind(8888)
        server.start(0) # autodetect cpu cores and fork one process per core
        db = motor.MotorClient().open_sync().proba
        print 'running on port %s' % options.port
        # Delayed initialization of settings
        app.settings['db'] = db # from this point on db is available as self.settings['db']
        tornado.ioloop.IOLoop.instance().start()

    except KeyboardInterrupt:
        tornado.ioloop.IOLoop.instance().stop()

推荐答案

如果我注释掉核心"和服务"导入，它会按预期工作.这些模块之一中的某些内容必须初始化单例事件循环(可能是间接的，例如通过创建全局 AsyncHTTPClient 实例).此警告是为了保护您免受在父进程中创建的这些对象在子进程中不起作用的事实.您必须找到创建这些对象的位置(遗憾的是没有很好的工具)并在 fork 之后移动它们，以便在子进程中创建它们.

It works as expected if I comment out the "core" and "services" imports. Something in one of those modules must be initializing the singleton event loop (perhaps indirectly, e.g. by creating a global AsyncHTTPClient instance). This warning is protecting you from the fact that these objects that were created in the parent process won't work in the child. You'll have to find the places where these objects are being created (unfortunately there aren't good tools for this) and move them after the fork so they are created in the child processes instead.

这篇关于运行多个龙卷风进程的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！