uWSGI懒应用程序和ThreadPool [英] uWSGI lazy-apps and ThreadPool
问题描述
使用以下命令运行uWSGI
Running uWSGI with
$ cat /etc/uwsgi/uwsgi.cfg
[uwsgi]
callable = app
socket = /var/run/arivale-service/uwsgi.sock
chmod-socket = 666
pidfile = /var/run/arivale-service/uwsgi.pid
master = true
enable-threads = true
single-interpreter = true
thunder-lock
need-app
processes = 4
在未启用lazy-apps
的情况下,对随后的终结点的呼叫的请求被挂起
Without lazy-apps
enabled, a request to the calling following endpoint hangs
import boto3
# ...irrelevant imports
from multiprocessing.dummy import Pool as ThreadPool
POOL = ThreadPool(6)
# ...irrelevant setup
def get_ecs_task_definitions(service_name):
ecs_service_name, _ = get_ecs_service_name_and_cluster(service_name)
def get_task_definition(task_definition_arn):
formatted_task_definition = {}
task_definition = ECS.describe_task_definition(taskDefinition=task_definition_arn)['taskDefinition']
# ...
# build formatted_task_definition from task_definition
# ...
return formatted_task_definition
task_definition_arns = ECS.list_task_definitions(
familyPrefix=ecs_service_name, status='ACTIVE')['taskDefinitionArns']
return POOL.map(get_task_definition, task_definition_arns)
@service.api('/api/services/<service_name>/ecs/task-definitions')
def get_task_definitions(service_name):
return {'service_name': service_name, 'task_definitions': get_ecs_task_definitions(service_name)}
NGINX正在平衡uWSGI应用程序,这就是error.log
(NGINX)中的说法:
NGINX is balancing uWSGI apps, and this is what it says in error.log
(NGINX):
Jun 10 03:54:26 93e04e04e2cf nginx_error: 2017/06/10 03:54:26 [error] 49#49: *33 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 172.16.254.95, server: localhost, request: "GET /api/services/data-analysis-service/ecs/task-definitions HTTP/1.1",upstream: "uwsgi://unix:/var/run/arivale-service/uwsgi.sock", host: "devops-service.arivale.com"
对端点的每个请求都挂起一个工作程序(以下是2个请求后 uwsgitop 的输出) :
Each request to the endpoint hangs a worker (below is the output of uwsgitop after 2 requests):
uwsgi-2.0.15 - Sat Jun 10 21:26:10 2017 - req: 0 - RPS: 0 - lq: 0 - tx: 0
node: localhost - cwd: /var/www/arivale-service/web - uid: 0 - gid: 0 - masterpid: 45
WID % PID REQ RPS EXC SIG STATUS AVG RSS VSZ TX ReSpwn HC RunT LastSpwn
1 0.0 74 0 0 0 0 busy 0ms 0 0 0 1 0 0 21:23:20
2 0.0 75 0 0 0 0 busy 0ms 0 0 0 1 0 0 21:23:20
3 0.0 76 0 0 0 0 idle 0ms 0 0 0 1 0 0 21:23:20
4 0.0 77 0 0 0 0 idle 0ms 0 0 0 1 0 0 21:23:20
启用lazy-apps
可解决此问题.谁能肯定地知道为什么?
Enabling lazy-apps
fixes the issue. Does anyone know with certainty why?
推荐答案
发生这种情况是因为uWSGI工作人员无法访问在主服务器中创建的线程池,请参见
This happened because uWSGI workers were unable to access the thread pool created in the master, see https://stackoverflow.com/a/39890941/2419669.
使用 @postfork 可以解决此问题:
global THREAD_POOL = None
@postfork
def _make_thread_pool():
global THREAD_POOL
THREAD_POOL = ThreadPool(8)
这篇关于uWSGI懒应用程序和ThreadPool的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!