Cron没有运行django命令 [英] Cron not running django command

问题描述

我有一个应该在特定时间每天运行的django脚本。我试图用 crontab 来达到这个目的。该脚本应该转储数据库，使用 gzip 将其归档并上传到 bitbucket 。

以下是我的 crontab 文件的相关部分：

  00 4 * * * root python /my_django_project_path/manage.py update_locations 
 47 16 * * * root python /my_django_project_path/manage.py database_bu 
  

当我执行 python /my_django_project_path/manage.py database_bu 时，它工作得很好。然而，crontab或者不执行它，或者一路上发生的事情。第一个crontab命令（ update_locations ）的执行效果非常好。

阅读

> 将命令更改为：

  47 16 * * * root（cd / my_django_project_path /& & python manage.py database_bu）
  

将命令更改为：

  47 16 * * * root / usr / bin / python /my_django_project_path/manage.py database_bu 
  
 $ b  将以下内容添加到我的脚本中（即使其他脚本没有它也能正常工作）： 
 ＃！/ usr / bin / python 
 
 $ from django.core.management import setup_environ 
导入设置
 setup_environ（设置）
  
 通过导出django项目设置的脚本：  
 
 
 /my_django_project_path/cron_command_executor.sh： 
 
 
  export DJANGO_SETTINGS_MODULE = my_django_project.settings 
 python manage.py $ {*} 
  
 crontab中的以下内容：
  47 16 * * * root ./my_django_project_path/cron_command_executor.sh database_bu 
  
 将用户更改为我的用户和Apache用户（ www-data ）。 
 
 
 
 我的crontab文件末尾有换行符。  
 $ b  更新： 
 
 
 
当执行 sudo su ，手动运行该命令不再有效。 
 $ b 
输出 tail -f / var / log / syslog 是：
  Mar 3 18:26:01 my-ip-address cron [726]：（system）RELOAD（/ etc / crontab）
 Mar 3 18:26:01 my-ip-address CRON [1184] :(根）CMD（python /my_django_project_path/manage.py database_bu）
  
 更新：  $ b 
我正在使用以下<$ c $ 
  machine bitbucket.org $ b $  .netrc  file防止git要求凭证： b登录myusername 
密码mypassword 
  
备份脚本的实际代码是：
 导入子流程
导入sh 
导入日期时间
从django.core导入gzip 
。 management.base import BaseCommand 
 $ b $ class Command（BaseCommand）：
 def handle（self，* args，** options）：
 execute_backup（）
 
 FILE_NAME ='some_file_name.sql'
 ARCHIV E_NAME ='some_archive_name.gz'
 REPO_NAME ='some_repo_name'
 GIT_USER ='some_git_username'＃你也需要在.netrc中改变它。 
 MYSQL_USER ='some_mysql_user'
 MYSQL_PASS ='some_mysql_pass'
 DATABASE_TO_DUMP ='SomeDatabase'＃您可以使用--all-databases，但要小心！它会转储一切！ 
 
 def dump_dbs_to_gzip（）：
＃转储参数。 
 args = [
'mysqldump'，'-u'，MYSQL_USER，'-p％s'％（MYSQL_PASS），
'--add-drop-database'，
 DATABASE_TO_DUMP，
] 
＃转储到文件。 
 dump_file = open（FILE_NAME，'w'）
 mysqldump_process = subprocess.Popen（args，stdout = dump_file）
 retcode = mysqldump_process.wait（）
 dump_file.close（） 
如果retcode> 0：
打印'备份错误'
＃压缩。 
 sql_file = open（FILE_NAME，'r'）
 gz_file = gzip.open（ARCHIVE_NAME，'wb'）
 gz_file.writelines（sql_file）
 gz_file.close（）
 sql_file.close（）
＃删除原始文件。 
 sh.rm（' -  f'，FILE_NAME）
 
 def clone_repo（）：
＃设置存储库位置。 
 repo_origin ='https：//[e-mail]/%s/%s.git'％（GIT_USER，GIT_USER，REPO_NAME）
 
克隆/ tmp中的存储库夹。 
 sh.cd（'/ tmp'）
 sh.rm（' -  rf'，REPO_NAME）
 sh.git.clone（repo_origin）
 sh.cd（REPO_NAME） 
 
 def commit_and_push（）：
＃提交并推送。 
 sh.git.add（'。'）
 sh.git.commit（m = datetime.datetime.now（）。strftime（％Y-％m-％d％H：％M ：'％s））
 sh.git.push（'origin'，'master'）
 sh.cd（'..'）
 sh.rm（' -  rf'， REPO_NAME）
 $ b $ def execute_backup（）：
 clone_repo（）
 dump_dbs_to_gzip（）
 commit_and_push（）
 $ b $ if if __name__ ==__main__ ：
 execute_backup（）
  
 更新： 
 
 
 
我设法使用克里斯克拉克的建议直接调用脚本而不是通过管理的.py 。但是，我仍然对引起此问题的原因感兴趣，因此赏金仍然可用。
 
 
 
 更新[已解决]： 
 
 
 
将以下行添加到 / etc / environment 并以我的用户帐户而非root用户身份运行它：
  PWD = / my_django_project_path / helpers / management / commands 
  
我仍然想知道为什么只有我的用户可以运行它，所以如果有人有解决方案，请捐助。
    / div> 
自  的某些版本适用于您，这意味着问题出在您的 cron environment ，或者按照您设置cron的方式，而不是使用脚本本身（例如要上传的文件大小或网络连接不会导致问题） 。
 
 
 
首先，您将脚本运行为 
 
 
 
 
 47 16 * * * root python /my_django_project_path/manage.py database_bu  
 
您正在为它提供一个用户名 root ，它与当前用户不是同一用户，而shell命令适用于当前用户。使用 sudo su 的命令不能从 root 用户运行，这表明您的root用户帐户不是无论如何正确配置。 FWIW，以root身份安排某些事情几乎总是可以避免的，因为它可能会导致文件权限问题。
 
 
 
因此，请尝试从当前用户调度您的cron作业。 / p> 
 
 
  47 16 * * * cd / my_django_project_path /&& python manage.py database_bu 
  
这可能仍然不能完全运行cron作业。在这种情况下，问题可能出现在两个地方 - 您的shell环境有一些cron环境中缺少的变量，或者您的 .netrc 文件未被正确读取/ b> 
 
 
根据我的经验，我发现 PATH 变量会导致最多的麻烦，所以在你的shell上运行 echo $ PATH ，如果你得到的路径值是 / some / path：/ some / other / path：/ more /路径/值，运行你的cron作业，如
  47 16 * * * export PATH =/ some / path：/ some / other / path：/ more / path / values&& cd / my_django_project_path /&& python manage.py database_bu 
  
如果不能解决问题，请检查所有环境变量。 / p> 
 
 
使用 printenv> 〜/ environment.txt 从一个普通的shell获得所有在shell中设置的环境变量。然后使用以下cron条目 * * * * * printenv> 〜/ cron_environment.txt 来识别cron环境中缺少的变量。或者，您可以使用脚本中的代码片段从脚本中获取环境值。 
 
 
  import os 
 os.system（printenv）
  
比较两者，找出不同的相关变量（如 HOME ），并尝试在脚本/ cron条目中使用相同的内容来检查它们是否正常工作。
 
 
 
如果事情仍然没有解决，那么我认为剩下的问题应该在 .netrc 中的bitbucket凭证中保存用户名和密码。内容 .netrc 可能在cron环境中不可用。  相反，create和为您的帐户设置ssh密钥对，并让备份发生在 ssh 而不是 https （如果你在这一步生成一个没有密码的ssh密钥，避免ssh-key的陷阱）。
 
 
一旦你设置了ssh密钥，你就必须相应地从 .git / config  origin_ssh 使用 git remote add origin_ssh url 请注意， https 回购网站的网址如 https ：//[e-mail]/user/repo.git ，而ssh则类似于 [e-mail]：user / repo.git 。
 
 
 
 PS： bitbucket 或更确切地说是 git 不是备份的理想解决方案，因此有更多的线程挂起以实现更好的备份策略。此外，在调试时，每分钟运行一次cron（ * * * * ），或以相似的低频率进行调试。 
 
    
 
 
 OP在评论中说，设置 PWD  
 $ b 
 
 
 PWD = / my_django_project_path / helpers / management /命令到/ etc / environment 
 
 
这是我之前的建议，shell中可用的环境变量之一不在cron环境中。 
 
 
 
一般而言，crown总是以减少的环境变量和权限运行，设置正确的变量将使cron正常工作。
 
 
 
另外，因为您正在使用 .netrc 文件进行权限设置，所以它对您的帐户特定，因此不适用于任何其他帐户（包括 sudo 帐户 root ），除非您在其他帐户中配置相同的设置。
 
I have a django script that should be run at a specified time every day. I am trying to achieve this using crontab. The script is supposed to dump the database, archive it using gzip and upload it to bitbucket.

The following is the relevant part of my crontab file:
00 4    * * *   root    python /my_django_project_path/manage.py update_locations
47 16   * * *   root    python /my_django_project_path/manage.py database_bu
When I execute python /my_django_project_path/manage.py database_bu it works perfectly fine. However crontab either does not execute it or something happens along the way. Even weirder, the first crontab command (update_locations) is executed perfectly fine.

Reading this question, I have tried the following, without success:

Changing the command to:
47 16   * * *   root    (cd /my_django_project_path/ && python manage.py database_bu)
Changing the command to:
47 16   * * *   root    /usr/bin/python /my_django_project_path/manage.py database_bu
Adding the following to my script (even though the other one works fine without it):
#!/usr/bin/python

from django.core.management import setup_environ
import settings
setup_environ(settings)
Running everything through a script that exports the django project settings:

/my_django_project_path/cron_command_executor.sh:
export DJANGO_SETTINGS_MODULE=my_django_project.settings 
python manage.py ${*}
The following in crontab:
47 16   * * *   root    ./my_django_project_path/cron_command_executor.sh database_bu
Changing the user to both my user and the Apache user (www-data).

I have a newline at the end of my crontab file.

UPDATE:

When doing sudo su, running the command manually no longer works. It gets stuck and doesn't do anything.

The output of tail -f /var/log/syslog is: 
Mar 3 18:26:01 my-ip-address cron[726]: (system) RELOAD (/etc/crontab) 
Mar 3 18:26:01 my-ip-address CRON[1184]: (root) CMD (python /my_django_project_path/manage.py database_bu)
UPDATE:

I am using the following .netrc file to prevent git asking for credentials:
machine bitbucket.org
    login myusername
    password mypassword
The actual code for the backup script is:
import subprocess
import sh
import datetime
import gzip
from django.core.management.base import BaseCommand

class Command(BaseCommand):
    def handle(self, *args, **options):
        execute_backup()

FILE_NAME = 'some_file_name.sql'
ARCHIVE_NAME = 'some_archive_name.gz'
REPO_NAME    = 'some_repo_name'
GIT_USER = 'some_git_username' # You'll need to change this in .netrc as well.
MYSQL_USER   = 'some_mysql_user'
MYSQL_PASS   = 'some_mysql_pass'
DATABASE_TO_DUMP = 'SomeDatabase' # You can use --all-databases but be careful with it! It will dump everything!.

def dump_dbs_to_gzip():
    # Dump arguments.
    args = [
        'mysqldump', '-u', MYSQL_USER, '-p%s' % (MYSQL_PASS),
        '--add-drop-database',
        DATABASE_TO_DUMP,
    ]
    # Dump to file.
    dump_file = open(FILE_NAME, 'w')
    mysqldump_process = subprocess.Popen(args, stdout=dump_file)
    retcode = mysqldump_process.wait()
    dump_file.close()
    if retcode > 0:
        print 'Back-up error'
    # Compress.
    sql_file = open(FILE_NAME, 'r')
    gz_file = gzip.open(ARCHIVE_NAME, 'wb')
    gz_file.writelines(sql_file)
    gz_file.close()
    sql_file.close()
    # Delete the original file.
    sh.rm('-f', FILE_NAME)

def clone_repo():
    # Set the repository location.
    repo_origin = 'https://[e-mail]/%s/%s.git' % (GIT_USER, GIT_USER, REPO_NAME)

    # Clone the repository in the /tmp folder.
    sh.cd('/tmp')
    sh.rm('-rf', REPO_NAME)
    sh.git.clone(repo_origin)
    sh.cd(REPO_NAME)

def commit_and_push():
    # Commit and push.
    sh.git.add('.')
    sh.git.commit(m=datetime.datetime.now().strftime("%Y-%m-%d %H:%M:%S"))
    sh.git.push('origin', 'master')
    sh.cd('..')
    sh.rm('-rf', REPO_NAME)

def execute_backup():
    clone_repo()
    dump_dbs_to_gzip()
    commit_and_push()

if __name__ == "__main__":
    execute_backup()
UPDATE:

I managed to fix it using Chris Clark's suggestion of calling the script directly rather than through manage.py. However, I am still interested in what is causing this issue so the bounty is still available.

UPDATE [SOLVED]:

Adding the following line to /etc/environment and running it as my user account rather than root fixed it:
PWD=/my_django_project_path/helpers/management/commands
I still wonder why only my user can run it so if anyone has the solution to that, please contribute.
解决方案
Since some version of python /my_django_project_path/manage.py database_bu works for you, it means the problem is with your cron environment, or in the way you have set up your cron and not with the script itself (as in the size of file to be uploaded or network connectivity is not causing the issue).

Firstly, you are running the script as 

  
47 16   * * *   root    python /my_django_project_path/manage.py database_bu
You are providing it a username root, which is not the same user as your current user, while the shell command worked for your current user. The fact that the same command doesn't run from root user using sudo su suggests that your root user account is not properly configured anyway. FWIW, scheduling something as root should almost always be avoided because it can lead to weird file permission issues.

So try scheduling your cron job as follows from that current user.
47 16 * * * cd /my_django_project_path/ && python manage.py database_bu
This may still not run the cron job completely. In which case, the problem could be at 2 places - your shell environment is having some variables that are missing from your cron environment, or your .netrc file is not being read properly for credentials, or both.

In my experience, I have found that PATH variable causes the most troubles, so run echo $PATH on your shell, and if the path value you get is /some/path:/some/other/path:/more/path/values, run your cron job like 
47 16 * * * export PATH="/some/path:/some/other/path:/more/path/values" && cd /my_django_project_path/ && python manage.py database_bu
If this doesn't work out, check all the environment variables next.

Use printenv > ~/environment.txt from a normal shell to get all the environment variables set in the shell. Then use the following cron entry * * * * * printenv > ~/cron_environment.txt to identify the missing variables from the cron environment. Alternatively, you can use the snippet in a script to get the value of environment from with the script
import os
os.system("printenv")
Compare the two, figure out any other relevant variables which are different (like HOME), and try using the same within the script/cron entry to check if they work or not.

If things still don't work out, then I think the remaining problem should be with your bitbucket credentials in .netrc in which saving the username and password. The contents .netrc might not be available in the cron environment.

Instead, create and set up an ssh keypair for your account and let the backup happen over ssh instead of https (Its better if you generate a ssh key without passphrase in this step, to avoid ssh-keys' gotchas).

Once you have setup the ssh keys, you will accordingly have to edit the existing origin url from .git/config file of your project root (or will have to add a new remote origin_ssh using git remote add origin_ssh url for the ssh protocol).

Note that https urls for the repo is like https://[e-mail]/user/repo.git while the ssh one is like [e-mail]:user/repo.git.

PS: bitbucket, or rather git is not the ideal solution for backups, there are tonnes of threads hanging around for better backup strategies. Also, while debugging, run your crons every minute (* * * * *), or at similarly low frequency for faster debugging.

Edit

OP says in the comment that setting the PWD variable worked for him.

  
PWD=/my_django_project_path/helpers/management/commands to /etc/environment
This is what I had suggested earlier, one of the environment variable available in the shell not being present in cron environment. 

In general, crown always runs with a reduced set of environment variable and permission, and setting the right variables will make cron work.

Also since you are using a .netrc file for permissions, it is specific to your account, and therefore that won't work with any other account (including the sudo account for root), unless you configure the same setting  in your other account as well.

                        
这篇关于Cron没有运行django命令的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！