Django REST Framework序列化POST速度很慢 [英] Django REST Framework Serialization POST is slow
问题描述
我正在Django 2.1.1和Python 3.6.5上运行,并且正在执行相当大的POST操作(32,000个JSON对象).我有以下内容:
I am running on Django 2.1.1 and Python 3.6.5 and am performing a reasonably large POST operation (32,000 JSON objects). I have the following:
型号:
class Data(models.Model):
investigation = models.ForeignKey(Investigation)
usage = models.FloatField()
sector = models.CharField(max_length=100, blank=False, default='')
cost = models.FloatField()
demand = models.FloatField()
序列化器:
class DataSerializer(serializers.ModelSerializer):
class Meta:
model = Data
fields = ('investigation', 'usage', 'sector', 'cost', 'demand')
查看:
class DataView(generics.CreateAPIView):
def create(self, request, pk, format=None):
data_serializer = DataSerializer(data=request.data, many=True)
if data_serializer.is_valid():
data_serializer.save()
问题出在is_valid()和save()步骤上,这两个步骤分别为32,000个对象中的每一个触发单独的查询.
The problems come at both the is_valid() and save() steps which each fire off a separate query for each of the 32,000 objects.
我花了很长时间研究这个问题,我猜is_valid()步很慢,因为N + 1查询问题,因为每次都要查询外键(尽管我可以这是错误的!),但我不知道如何在此框架中实现prefetch_related方法.
I've spent a long time looking into the issue and I'm guessing that the is_valid() step is slow because of the N+1 query problem since the foreign key is being looked up each time (although I could be wrong about this!) but I have no idea how to implement the prefetch_related method in this framework.
save()步骤(这是最慢的部分)显然需要在一个查询(可能是bulk_create)中完成,但是我找不到在哪里添加bulk_create步骤.我已阅读
The save() step (which is the slowest part) obviously needs to be done in one query (probably a bulk_create) but I can't find where to add the bulk_create step in. I've read this question but am still none the wiser from the answer. I tried to create a ListSerializer as the question suggests but the objects still seemed to be serialized one by one.
任何指针将不胜感激.
推荐答案
一种可能的解决方案是在使用序列化程序验证数据后执行Django ORM bulk_create()
.您的视图将如下所示:
One possible solution is to perform a Django ORM bulk_create()
after you validate the data using your serializer. Your view will then look something like this:
class DataView(generics.CreateAPIView):
def create(self, request, pk, format=None):
data_serializer = DataSerializer(data=request.data, many=True)
if data_serializer.is_valid():
data_objects = []
for data_object_info in data_serializer.validated_data:
data_objects.append(Data(**data_object_info))
Data.objects.bulk_create(data_objects)
或仅以下一项,如果您想要单线:
or just the following, if you want a one-liner:
Data.objects.bulk_create([Data(**params) for params in data_serializer.validated_data])
如果不想使视图混乱,则可以编写一个执行验证(使用序列化程序)和创建的类或方法.然后,您可以在视图内部使用它.
If you don't want to clutter your view, then you can write a class or method that performs the validation (using the serializer) and creation. You can then use this inside the view.
这篇关于Django REST Framework序列化POST速度很慢的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!