Django REST Framework序列化POST速度很慢 [英] Django REST Framework Serialization POST is slow

问题描述

我正在Django 2.1.1和Python 3.6.5上运行，并且正在执行相当大的POST操作(32,000个JSON对象).我有以下内容:

I am running on Django 2.1.1 and Python 3.6.5 and am performing a reasonably large POST operation (32,000 JSON objects). I have the following:

型号:

class Data(models.Model):
    investigation = models.ForeignKey(Investigation)
    usage = models.FloatField()
    sector = models.CharField(max_length=100, blank=False, default='')
    cost = models.FloatField()
    demand = models.FloatField()

序列化器:

class DataSerializer(serializers.ModelSerializer):
    class Meta:
        model = Data
        fields = ('investigation', 'usage', 'sector', 'cost', 'demand')

查看:

class DataView(generics.CreateAPIView):
    def create(self, request, pk, format=None):
        data_serializer = DataSerializer(data=request.data, many=True)
        if data_serializer.is_valid():
            data_serializer.save()

问题出在is_valid()和save()步骤上，这两个步骤分别为32,000个对象中的每一个触发单独的查询.

The problems come at both the is_valid() and save() steps which each fire off a separate query for each of the 32,000 objects.

我花了很长时间研究这个问题，我猜is_valid()步很慢，因为N + 1查询问题，因为每次都要查询外键(尽管我可以这是错误的！)，但我不知道如何在此框架中实现prefetch_related方法.

I've spent a long time looking into the issue and I'm guessing that the is_valid() step is slow because of the N+1 query problem since the foreign key is being looked up each time (although I could be wrong about this!) but I have no idea how to implement the prefetch_related method in this framework.

save()步骤(这是最慢的部分)显然需要在一个查询(可能是bulk_create)中完成，但是我找不到在哪里添加bulk_create步骤.我已阅读

The save() step (which is the slowest part) obviously needs to be done in one query (probably a bulk_create) but I can't find where to add the bulk_create step in. I've read this question but am still none the wiser from the answer. I tried to create a ListSerializer as the question suggests but the objects still seemed to be serialized one by one.

任何指针将不胜感激.

推荐答案

一种可能的解决方案是在使用序列化程序验证数据后执行Django ORM bulk_create().您的视图将如下所示:

One possible solution is to perform a Django ORM bulk_create() after you validate the data using your serializer. Your view will then look something like this:

class DataView(generics.CreateAPIView):
    def create(self, request, pk, format=None):
        data_serializer = DataSerializer(data=request.data, many=True)
        if data_serializer.is_valid():
            data_objects = []
            for data_object_info in data_serializer.validated_data:
                data_objects.append(Data(**data_object_info))
            Data.objects.bulk_create(data_objects)

或仅以下一项，如果您想要单线:

or just the following, if you want a one-liner:

Data.objects.bulk_create([Data(**params) for params in data_serializer.validated_data])

如果不想使视图混乱，则可以编写一个执行验证(使用序列化程序)和创建的类或方法.然后，您可以在视图内部使用它.

If you don't want to clutter your view, then you can write a class or method that performs the validation (using the serializer) and creation. You can then use this inside the view.

这篇关于Django REST Framework序列化POST速度很慢的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！