python - 请教使用supervisor启动pyspider运行中phantomjs进程数量过多问题

问题描述

问 题

下面是supervisor的配置

[program:pyspider]
command=/usr/local/bin/pyspider -c /root/config.json
autostart=true
autorestart=true
user=root
directory=/root

不使用supervisor来启动pyspider,phantomjs进程为2个.
使用supervisor后,phantomjs进程超过6个以上,并且在代码界面进行测试也会卡死,只能kill掉pyspider和phantomjs才行.
环境是Ubuntu14.04X64
python2.7.6
phantomjs1.9.0
vps内存768M

解决方案

[group:pyspider]
program=pyspider-webui,pyspider-scheduler,pyspider-processor,pyspider-result_worker,pyspider-fetcher,pyspider-phantomjs
priority=999

[program:pyspider-webui]
command=/usr/local/bin/pyspider/run.py -c /root/config.json webui
directory=/root
autostart=true
autorestart=true
priority=905
user=root

[program:pyspider-scheduler]
command=/usr/local/bin/pyspider/run.py -c /root/config.json scheduler
directory=/root
autostart=true
autorestart=true
priority=900
user=root

[program:pyspider-processor]
command=/usr/local/bin/pyspider/run.py -c /root/config.json processor
directory=/root
autostart=true
autorestart=true
priority=903
user=root

[program:pyspider-result_worker]
command=/usr/local/bin/pyspider/run.py -c /root/config.json result_worker
directory=/root
autostart=true
autorestart=true
priority=904
user=root

[program:pyspider-fetcher]
command=/usr/local/bin/pyspider/run.py -c /root/config.json --phantomjs-proxy="localhost:25555" fetcher
directory=/root
autostart=true
autorestart=true
priority=902
user=root

[program:pyspider-phantomjs]
command=/usr/local/bin/pyspider/run.py -c /root/config.json phantomjs
directory=/root
autostart=true
autorestart=true
priority=901
user=root

这篇关于python - 请教使用supervisor启动pyspider运行中phantomjs进程数量过多问题的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！