ModelSerializer在Django REST框架中非常慢 [英] ModelSerializer is extremely slow in Django REST framework

问题描述

我正在使用Django REST框架为我的API，昨天我想看看它如何适用于大数据。我发现有关如何配置您的请求的本教程 （由汤姆·克里斯蒂写的），我发现，对于10,000位用户，我的请求是惊人的2:20分钟。

大部分时间都用于序列化对象（大约65％），所以我想知道我可以做些什么来加快速度？

我的用户模型实际上是扩展了默认的django模型，所以使用。values（）不起作用，因为我还没有得到嵌套模型（即使是更快的）。

任何帮助将不胜感激：）

编辑

在检索我的查询集时，我已经使用了.select_related（），它已经改善了我的时间，但只有几秒钟。总查询数是10，所以我的问题不在数据库访问。

另外，我使用.defer（），为了避免字段我不需要这个请求。这也提供了一个小的改进，但还不够。

编辑＃2

模型

  from django.contrib.auth.models import User $来自django.db.models的b $ b导入来自django.db.models的OneToOneField 
 import usK从
 
从userena.models导入UserenaLanguageBaseProfile 
 from django_extensions.db.fields import CreationDateTimeField 
来自django_extensions.db.fields import modificationDateTimeField 
 
 from mycompany.models import MyCompany 
 
 
 class UserProfile（UserenaLanguageBaseProfile）：
 user = OneToOneField （User，related_name ='user_profile'）
 company = ForeignKey（MyCompany）
 created = CreationDateTimeField（_（'created'））
 modified = ModificationDateTimeField（_（'modified'））
  

序列化器

  from django.contrib.auth.models import用户
 
 from re st_framework import serializers 
 
 from accounts.models import UserProfile 
 
 
类UserSerializer（serializers.ModelSerializer）：
 last_login = serializers.ReadOnlyField（）
 date_joined = serializers.ReadOnlyField（）
 is_active = serializers.ReadOnlyField（）
 
 class Meta：
 model =用户
 fields =（
' id'，
'last_login'，
'username'，
'first_name'，
'last_name'，
'email'，
'is_active' ，
'date_joined'，
）
 
 
 class UserProfileSerializer（serializers.ModelSerializer）：
 user = UserSerializer（）
 
 class Meta：
 model = UserProfile 
 fields =（
'id'，
'user'，
'mugshot'，
'language' 
）
  

视图

  class UserProfileList（generics.GenericAPIView，
 mixins.ListModelMixin，
 mixins.CreateModelMixin）：
 
 serializer_class = UserProfileSerializer 
 permission_classes =（UserPermissions，）
 
 def get_queryset（self）：
 company = self.request.user.user_profile.company 
 return UserProfile.objects.select_related（）。filter（company = company）
 
 @etag（etag_func = UserListKeyConstructor（））
 def get（self，request，* args，** kwargs）：
 return self.list（request，* args，** kwargs） 
  

解决方案

几乎总是性能问题来自N + 1查询。这通常是因为您引用相关模型，并且生成每个对象每个关系的单个查询以获取信息。您可以使用 .select_related 和 .prefetch_related 在您的 get_queryset 方法，如我的其他堆栈溢出答案所述。

与 Django提供数据库优化的提示相同适用于Django REST框架，所以我建议您查看这些。

在串行化期间看到性能问题的原因是因为Django在查询数据库。

I am using Django REST framework for my API and yesterday I wanted to see how it works for large data. I found this tutorial about how to profile your requests (written by Tom Christie) and I discovered that for 10.000 users, my request was taking an astonishing 2:20 minutes.

Most of the time was being spent on serializing the objects (around 65%) so I was wondering what can I do to speed things up ?

My user model is actually extending the default django model, so using .values() does not work, because I am not also getting the nested model (even though it is a LOT faster).

Any help would be greatly appreciated :)

Edit

I am already using .select_related() when retrieving my queryset, and it has improved my time, but only by a few seconds. The number of total queries is 10, so my problem is not with the database access.

Also, I am using .defer(), in order to avoid fields that I don't need in this request. That also provided a small improvement, but not enough.

Edit #2

Models

from django.contrib.auth.models import User
from django.db.models import OneToOneField
from django.db.models import ForeignKey

from userena.models import UserenaLanguageBaseProfile
from django_extensions.db.fields import CreationDateTimeField
from django_extensions.db.fields import ModificationDateTimeField

from mycompany.models import MyCompany


class UserProfile(UserenaLanguageBaseProfile):
    user = OneToOneField(User, related_name='user_profile')
    company = ForeignKey(MyCompany)
    created = CreationDateTimeField(_('created'))
    modified = ModificationDateTimeField(_('modified'))

Serializers

from django.contrib.auth.models import User

from rest_framework import serializers

from accounts.models import UserProfile


class UserSerializer(serializers.ModelSerializer):
    last_login = serializers.ReadOnlyField()
    date_joined = serializers.ReadOnlyField()
    is_active = serializers.ReadOnlyField()

    class Meta:
        model = User
        fields = (
            'id',
            'last_login',
            'username',
            'first_name',
            'last_name',
            'email',
            'is_active',
            'date_joined',
        )


class UserProfileSerializer(serializers.ModelSerializer):
    user = UserSerializer()

    class Meta:
        model = UserProfile
        fields = (
            'id',
            'user',
            'mugshot',
            'language',
        )

Views

class UserProfileList(generics.GenericAPIView,
                      mixins.ListModelMixin,
                      mixins.CreateModelMixin):

    serializer_class = UserProfileSerializer
    permission_classes = (UserPermissions, )

    def get_queryset(self):
        company = self.request.user.user_profile.company
        return UserProfile.objects.select_related().filter(company=company)

    @etag(etag_func=UserListKeyConstructor())
    def get(self, request, *args, **kwargs):
        return self.list(request, *args, **kwargs)

解决方案

Almost always the performance issues come from N+1 queries. This is usually because you are referencing related models, and a single query per relationship per object is generated to get the information. You can improve this by using .select_related and .prefetch_related in your get_queryset method, as described in my other Stack Overflow answer.

The same tips that Django provides on database optimization also applies to Django REST framework, so I would recommend looking into those as well.

The reason why you are seeing the performance issues during serialization is because that is when Django makes the queries to the database.

这篇关于ModelSerializer在Django REST框架中非常慢的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！