Cron 没有运行 django 命令 [英] Cron not running django command

问题描述

我有一个 django 脚本，它应该每天在指定的时间运行.我正在尝试使用 crontab 来实现这一点.该脚本应该转储数据库，使用 gzip 将其存档并将其上传到 bitbucket.

I have a django script that should be run at a specified time every day. I am trying to achieve this using crontab. The script is supposed to dump the database, archive it using gzip and upload it to bitbucket.

以下是我的crontab文件的相关部分:

The following is the relevant part of my crontab file:

00 4    * * *   root    python /my_django_project_path/manage.py update_locations
47 16   * * *   root    python /my_django_project_path/manage.py database_bu

当我执行 python/my_django_project_path/manage.py database_bu 时，它运行良好.然而，crontab 要么不执行它，要么沿途发生了一些事情.更奇怪的是，第一个 crontab 命令 (update_locations) 执行得非常好.

When I execute python /my_django_project_path/manage.py database_bu it works perfectly fine. However crontab either does not execute it or something happens along the way. Even weirder, the first crontab command (update_locations) is executed perfectly fine.

阅读这个问题，我尝试了以下方法，但没有成功:

Reading this question, I have tried the following, without success:

将命令更改为:

47 16   * * *   root    (cd /my_django_project_path/ && python manage.py database_bu)

将命令更改为:

47 16   * * *   root    /usr/bin/python /my_django_project_path/manage.py database_bu

将以下内容添加到我的脚本中(即使另一个脚本没有它也能正常工作):

#!/usr/bin/python

from django.core.management import setup_environ
import settings
setup_environ(settings)

通过导出 django 项目设置的脚本运行所有内容:

/my_django_project_path/cron_command_executor.sh:

/my_django_project_path/cron_command_executor.sh:

export DJANGO_SETTINGS_MODULE=my_django_project.settings 
python manage.py ${*}

crontab 中的以下内容:

The following in crontab:

47 16   * * *   root    ./my_django_project_path/cron_command_executor.sh database_bu

将用户更改为我的用户和 Apache 用户 (www-data).

Changing the user to both my user and the Apache user (www-data).

我的 crontab 文件末尾有一个换行符.

更新:

在执行 sudo su 时，手动运行命令不再有效.它卡住了，什么也不做.

When doing sudo su, running the command manually no longer works. It gets stuck and doesn't do anything.

tail -f/var/log/syslog 的输出是:

Mar 3 18:26:01 my-ip-address cron[726]: (system) RELOAD (/etc/crontab) 
Mar 3 18:26:01 my-ip-address CRON[1184]: (root) CMD (python /my_django_project_path/manage.py database_bu)

更新:

我正在使用以下 .netrc 文件来防止 git 要求提供凭据:

I am using the following .netrc file to prevent git asking for credentials:

machine bitbucket.org
    login myusername
    password mypassword

备份脚本的实际代码是:

The actual code for the backup script is:

import subprocess
import sh
import datetime
import gzip
from django.core.management.base import BaseCommand

class Command(BaseCommand):
    def handle(self, *args, **options):
        execute_backup()

FILE_NAME = 'some_file_name.sql'
ARCHIVE_NAME = 'some_archive_name.gz'
REPO_NAME    = 'some_repo_name'
GIT_USER = 'some_git_username' # You'll need to change this in .netrc as well.
MYSQL_USER   = 'some_mysql_user'
MYSQL_PASS   = 'some_mysql_pass'
DATABASE_TO_DUMP = 'SomeDatabase' # You can use --all-databases but be careful with it! It will dump everything!.

def dump_dbs_to_gzip():
    # Dump arguments.
    args = [
        'mysqldump', '-u', MYSQL_USER, '-p%s' % (MYSQL_PASS),
        '--add-drop-database',
        DATABASE_TO_DUMP,
    ]
    # Dump to file.
    dump_file = open(FILE_NAME, 'w')
    mysqldump_process = subprocess.Popen(args, stdout=dump_file)
    retcode = mysqldump_process.wait()
    dump_file.close()
    if retcode > 0:
        print 'Back-up error'
    # Compress.
    sql_file = open(FILE_NAME, 'r')
    gz_file = gzip.open(ARCHIVE_NAME, 'wb')
    gz_file.writelines(sql_file)
    gz_file.close()
    sql_file.close()
    # Delete the original file.
    sh.rm('-f', FILE_NAME)

def clone_repo():
    # Set the repository location.
    repo_origin = 'https://[e-mail]/%s/%s.git' % (GIT_USER, GIT_USER, REPO_NAME)

    # Clone the repository in the /tmp folder.
    sh.cd('/tmp')
    sh.rm('-rf', REPO_NAME)
    sh.git.clone(repo_origin)
    sh.cd(REPO_NAME)

def commit_and_push():
    # Commit and push.
    sh.git.add('.')
    sh.git.commit(m=datetime.datetime.now().strftime("%Y-%m-%d %H:%M:%S"))
    sh.git.push('origin', 'master')
    sh.cd('..')
    sh.rm('-rf', REPO_NAME)

def execute_backup():
    clone_repo()
    dump_dbs_to_gzip()
    commit_and_push()

if __name__ == "__main__":
    execute_backup()

更新:

我设法使用 Chris Clark 建议直接调用脚本而不是通过 manage.py 修复它.但是，我仍然对导致此问题的原因感兴趣，因此赏金仍然可用.

I managed to fix it using Chris Clark's suggestion of calling the script directly rather than through manage.py. However, I am still interested in what is causing this issue so the bounty is still available.

更新 [已解决]:

将以下行添加到 /etc/environment 并以我的用户帐户而不是 root 运行它来修复它:

Adding the following line to /etc/environment and running it as my user account rather than root fixed it:

PWD=/my_django_project_path/helpers/management/commands

我仍然想知道为什么只有我的用户可以运行它，所以如果有人有解决方案，请贡献.

I still wonder why only my user can run it so if anyone has the solution to that, please contribute.

推荐答案

由于 python/my_django_project_path/manage.py database_bu 的 some version 对你有用，这意味着问题在于您的 cron 环境，或者您设置 cron 的方式而不是脚本本身(因为要上传的文件大小或网络连接不会导致问题).

Since some version of python /my_django_project_path/manage.py database_bu works for you, it means the problem is with your cron environment, or in the way you have set up your cron and not with the script itself (as in the size of file to be uploaded or network connectivity is not causing the issue).

首先，您正在运行脚本

47 16 * * * root python/my_django_project_path/manage.py database_bu

47 16 * * * root python /my_django_project_path/manage.py database_bu

您为其提供了一个用户名 root，该用户名与您当前的用户不同，而 shell 命令为您的当前用户工作.同一命令不能从 root 用户使用 sudo su 运行的事实表明您的 root 用户帐户没有正确配置.FWIW，几乎总是应该避免以 root 身份调度某些东西，因为它会导致奇怪的文件权限问题.

You are providing it a username root, which is not the same user as your current user, while the shell command worked for your current user. The fact that the same command doesn't run from root user using sudo su suggests that your root user account is not properly configured anyway. FWIW, scheduling something as root should almost always be avoided because it can lead to weird file permission issues.

因此，请尝试按照该当前用户的以下方式安排您的 cron 作业.

So try scheduling your cron job as follows from that current user.

47 16 * * * cd /my_django_project_path/ && python manage.py database_bu

这可能仍然无法完全运行 cron 作业.在这种情况下，问题可能出在 2 个地方 - 您的 shell 环境中缺少一些 cron 环境中缺少的变量，或者您的 .netrc 文件没有正确读取凭据，或者两者都有.

This may still not run the cron job completely. In which case, the problem could be at 2 places - your shell environment is having some variables that are missing from your cron environment, or your .netrc file is not being read properly for credentials, or both.

根据我的经验，我发现PATH变量最麻烦，所以在你的shell上运行echo $PATH，如果你得到的路径值是<代码>/some/path:/some/other/path:/more/path/values，像

In my experience, I have found that PATH variable causes the most troubles, so run echo $PATH on your shell, and if the path value you get is /some/path:/some/other/path:/more/path/values, run your cron job like

47 16 * * * export PATH="/some/path:/some/other/path:/more/path/values" && cd /my_django_project_path/ && python manage.py database_bu

如果这不起作用，接下来检查所有环境变量.

If this doesn't work out, check all the environment variables next.

使用 printenv >~/environment.txt 从普通shell 获取shell 中设置的所有环境变量.然后使用下面的 cron 条目 * * * * * printenv >~/cron_environment.txt 来识别 cron 环境中缺失的变量.或者，您可以使用脚本中的代码段从脚本中获取环境值

Use printenv > ~/environment.txt from a normal shell to get all the environment variables set in the shell. Then use the following cron entry * * * * * printenv > ~/cron_environment.txt to identify the missing variables from the cron environment. Alternatively, you can use the snippet in a script to get the value of environment from with the script

import os
os.system("printenv")

比较两者，找出任何其他不同的相关变量(如 HOME)，并尝试在脚本/cron 条目中使用相同的变量来检查它们是否有效.

Compare the two, figure out any other relevant variables which are different (like HOME), and try using the same within the script/cron entry to check if they work or not.

如果事情仍然没有解决，那么我认为剩下的问题应该是您在 .netrc 中的 bitbucket 凭据，其中保存了用户名和密码.内容 .netrc 在 cron 环境中可能不可用.

If things still don't work out, then I think the remaining problem should be with your bitbucket credentials in .netrc in which saving the username and password. The contents .netrc might not be available in the cron environment.

相反，创建并设置 ssh 密钥对 为您的帐户并让备份通过 ssh 而不是 https(如果您在此步骤中生成没有密码的 ssh 密钥会更好，以避免 ssh-keys 的陷阱).

Instead, create and set up an ssh keypair for your account and let the backup happen over ssh instead of https (Its better if you generate a ssh key without passphrase in this step, to avoid ssh-keys' gotchas).

一旦您设置了 ssh 密钥，您将相应地必须从项目根目录的 .git/config 文件中编辑现有的原始 url(或者必须添加一个新的远程 origin_ssh 使用 git remote add origin_ssh url 作为 ssh 协议).

Once you have setup the ssh keys, you will accordingly have to edit the existing origin url from .git/config file of your project root (or will have to add a new remote origin_ssh using git remote add origin_ssh url for the ssh protocol).

请注意，repo 的 https url 类似于 https://[e-mail]/user/repo.git 而 ssh 则类似于 [e-mail]:user/repo.git.

Note that https urls for the repo is like https://[e-mail]/user/repo.git while the ssh one is like [e-mail]:user/repo.git.

PS:bitbucket，或者更确切地说 git 不是备份的理想解决方案，有大量线程在等待更好的备份策略.此外，在调试时，每分钟运行一次 cron (* * * * *)，或以类似的低频率运行以加快调试速度.

PS: bitbucket, or rather git is not the ideal solution for backups, there are tonnes of threads hanging around for better backup strategies. Also, while debugging, run your crons every minute (* * * * *), or at similarly low frequency for faster debugging.

编辑

OP 在评论中说设置 PWD 变量对他有用.

OP says in the comment that setting the PWD variable worked for him.

PWD=/my_django_project_path/helpers/management/commands to/etc/environment

PWD=/my_django_project_path/helpers/management/commands to /etc/environment

这是我之前建议的，shell 中可用的环境变量之一在 cron 环境中不存在.

This is what I had suggested earlier, one of the environment variable available in the shell not being present in cron environment.

一般来说，crown 总是在减少环境变量和权限的情况下运行，设置正确的变量将使 cron 工作.

In general, crown always runs with a reduced set of environment variable and permission, and setting the right variables will make cron work.

此外，由于您使用 .netrc 文件获取权限，它特定于您的帐户，因此不适用于任何其他帐户(包括 sudo> root 的帐户)，除非您也在其他帐户中配置了相同的设置.

Also since you are using a .netrc file for permissions, it is specific to your account, and therefore that won't work with any other account (including the sudo account for root), unless you configure the same setting in your other account as well.

这篇关于Cron 没有运行 django 命令的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！