Django夹具加载非常慢 [英] Django fixture loading very slow

问题描述

我正在尝试使用 2 组夹具提供初始数据.第一种夹具格式如下所示.

I'm trying to provide initial data using 2 sets of fixtures. The first fixture format looks like this.

  {
    "pk": 1,
    "model": "data.Person",
    "fields": {
      "full": "Anna-Varney",
      "num": "I",
      "short": "Anna-Varney"
    }
  },

我首先加载它，大约 1-2 小时后加载良好.我的 movie.json 格式如下所示:

And I load it in first, and it loads in fine in roughly 1-2 hours. My movie.json format looks like this:

  {
    "pk": 1,
    "model": "data.Film",
    "fields": {
      "date": "2005-08-01",
      "rating": 8.3,
      "actors": [
        [
          "Anna-Varney"
        ]
      ],
      "name": "Like a Corpse Standing in Desperation (2005) (V)"
    }
  },

加载电影装置花了很长时间，目前已经 20 小时了，我的电脑在运行时速度很慢.我在 2 个月前加载了类似的装置，除了我使用 MySQL(我现在使用 Postgres)并且我在我的模型中添加了日期字段.以前把电影装置加载到我的旧 MySQL 数据库时，只花了 2-3 个小时.有没有办法确定夹具加载部件处于哪个步骤或它是否已冻结?

And loading the movies fixture in has taken an extremely long time, it's currently 20 hrs in and my computer is sluggish while it is running. I loaded similar fixtures 2 months ago, except I used MySQL (I'm using Postgres now) and that I've added the date field in my model. When loading the movies fixture into my old MySQL database in the past, it only took 2-3 hours. Is there a way to determine what step the fixture loading part is in or if it has frozen?

作为参考，我的模型是:

For reference my models are:

class PersonManager(models.Manager):
    def get_by_natural_key(self, full):
        return self.get(full=full)

class Person(models.Model):
    objects = PersonManager()
    full = models.CharField(max_length=100,unique = True)
    short = models.CharField(max_length=100)
    num = models.CharField(max_length=5)
    def natural_key(self):
        return (self.full,)

    def __unicode__(self):
        return self.full


class Film(models.Model):
    name = models.TextField()
    date = models.DateField()
    rating = models.DecimalField(max_digits=3 , decimal_places=1)
    actors = models.ManyToManyField('Person')

    def __unicode__(self):
        return self.name

推荐答案

如果您通过命令行加载设备:

If you are loading your fixtures via the command line:

python manage.py loaddata --database=MY_DB_LABEL fixtures/my_fixture.json;

或者可能通过 shell 以编程方式:

or perhaps programmatically through the shell:

os.system('python manage.py loaddata --database=%s fixtures/my_fixture.json;' % MY_DB_LABEL)

夹具加载将慢.(我还没有调查原因.大概是有很多不必要的中间数据库保存.)

Fixture loading will be SLOW. (I have not investigated why. Presumably, there are many unnecessary intermediate database saves being made.)

解决方案:切换到使用单个事务通过 python 以编程方式加载您的装置:

SOLUTION: Switch to loading your fixtures programatically via python using a single transaction:

from django.db import transaction
from django.core.management import call_command

with transaction.atomic(using=MY_DB_LABEL):
    call_command('loaddata', 'fixtures/my_fixture.json', database=MY_DB_LABEL)
    call_command('loaddata', 'fixtures/my_other_fixture.json', database=MY_DB_LABEL)

夹具加载将显着地加速.

请注意，这里的 database 和 using 参数是可选的.如果您使用的是单个数据库，则它们是不必要的.但是如果你像我一样使用多个数据库，你可能想用它来确保夹具数据加载到哪个数据库中.

Note that the database and using parameters here are optional. If you are using a single database, they are unnecessary. But if you are using multiple databases like me, you will probably want to use it to ensure which database the fixture data is loaded into.

这篇关于Django夹具加载非常慢的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！