在Cron工作的Scrapy爬行物 [英] Scrapy crawler in Cron job

问题描述

我想从cron工作执行我的scrapy crawler。

我创建bash文件getdata.sh其中scrapy项目位于它的蜘蛛

 ＃！/ bin / bash 
 cd / myfolder / crawlers / 
 scrapy crawl my_spider_name 
  
 
 
 
我的crontab看起来像这样，我想每5分钟执行一次
  * / 5 * * * sh /myfolder/crawlers/getdata.sh 
  
 
 $ b b 
但它不工作，什么错，我的错误在哪里？ 
 
 
 
当我从终端执行我的bash文件sh /myfolder/crawlers/getdata.sh它工作正常
解决方案
我在bash文件中解决了这个问题，包括PATH。
 ＃！/ bin / bash 
 
 cd / myfolder / crawlers / 
 PATH = $ PATH：/ usr / local / bin 
 export PATH 
 scrapy crawl my_spider_name 
  
 
I want to execute my scrapy crawler from cron job .

i create bash file getdata.sh where scrapy project is located with it's spiders
#!/bin/bash
cd /myfolder/crawlers/
scrapy crawl my_spider_name
My crontab looks like this , I want to execute it in every 5 minute
 */5 * * * * sh /myfolder/crawlers/getdata.sh 
but it don't works , whats wrong , where is my error ? 

when I execute my bash file from terminal sh /myfolder/crawlers/getdata.sh it works fine
解决方案
I solved this problem including PATH into bash file 
#!/bin/bash

cd /myfolder/crawlers/
PATH=$PATH:/usr/local/bin
export PATH
scrapy crawl my_spider_name


                        
这篇关于在Cron工作的Scrapy爬行物的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！