Django REST Serializer做N + 1数据库调用多个嵌套关系，3个级别 [英] Django REST Serializer doing N+1 database calls for multiple nested relationship, 3 levels

问题描述

我有一种情况，我的模型有外键关系：

 ＃models.py 
 class Child （models.Model）：
 parent = models.ForeignKey（Parent，）
 
 class父（models.Model）：
 pass 
  

和我的序列化器：

  class ParentSerializer serializer.ModelSerializer）：
 child = serializers.SerializerMethodField（'get_children_ordered'）
 
 def get_children_ordered（self，parent）：
 queryset = Child.objects.filter（parent = parent ）.select_related（'parent'）
 serialized_data = ChildSerializer（queryset，many = True，read_only = True，context = self.context）
 return serialized_data.data 
 
 class Meta ：
 model = Parent 
  

当我在N个父母的视图中调用Parent时， Django在抓取孩子时，在序列化器内部执行N个数据库调用。有没有办法让所有的父母的所有孩子最小化数据库调用的数量？

我已经尝试过，但似乎并没有解决我的问题：

  class ParentList（generics.ListAPIView）：
 
 def get_queryset（self）：
 queryset = Parent.objects.prefetch_related（'child'）
 return queryset 
 
 serializer_class = ParentSerializer 
 permission_classes =（permissions.IsAuthenticated，）
  

编辑

我已经更新了下面的代码，以反映Alex的反馈....解决了N + 1的一个嵌套关系。

 ＃serializer.py 
 class ParentSerializer（serializer.ModelSerializer）：
 child = serializers.SerializerMethodField 'get_children_ordered'）
 
 def get_children_ordered（self，parent）：
＃all（）调用应该命中缓存
 serialized_data = ChildSerializer（parent.child.all（） many = True，read_only = True，context = self.context）
返回serialized_data.data 
 
 class Meta：
 model = Parent 
 
＃views.py 
 class ParentList（generics.ListAPIView）：
 
 def get_queryset（self）：
 children = Prefetch（'child'，queryset = Child.objects.select_related（'parent'））
 queryset = Parent.objects.prefetch_related（children）
 return queryset 
 
 serializer_class = ParentSerializer 
 permission_classes =（permissions.IsAuthenticated，）
  

现在我再说一个模型，这是一个孙子：

 ＃models.py 
 class GrandChild（models.Model）：
 parent = models.ForeignKey（Child，）
 
 class Child（models.Model）：
 parent = models.ForeignKey（Parent ，）
 
 class父（models.Model）：
 pass 
  

如果我将以下内容放在我的 views.py 中，为父 queryset ：

查询et = Parent.objects.prefetch_related（children，'children__grandchildren'）

看起来不像那些孙子被带入ChildSerializer，因此，我再次运行另一个N + 1问题。任何想法在这一个？

编辑2

也许这将提供清晰度...也许我仍然在N + 1数据库调用中的原因是因为我的孩子和孙子类都是多态的.... ie

 $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ b pass 
 
 class GrandDaughter（GrandChild）：
 pass 
 
 class Child（PolymorphicModel）：
 parent = models.ForeignKey（Parent，）
 
 class Son（Child）：
 pass 
 
 class女儿（Child）：
 pass 
 
 class父（models.Model ）：
 pass 
  

，我的序列化程序看起来更像这样：

 ＃serializer.py 
 class ChildSerializer（serializer.ModelSerializer）：
 grandchild = serializers.SerializerMethodField（'get_children_ordered'）
 
 def to_representation（self，value）：
如果isinstance（value，Son）：
返回SonSerializer（value，context = self.context）.to_representation（value）
 if isinstance（value， ）
返回DaughterSerializer（value，context = self.context）.to_representation（value）
 
 class Meta：
 model = Child 
 
 class ParentSerializer （serializer.ModelSerializer）：
 child = serializers.SerializerMethodField（'get_children_ordered'）
 
 def get_children_ordered（self，parent）：
 queryset = Child.objects.filter（parent =父项）.select_related（'parent'）
 serialized_data = ChildSerializer（queryset，many = True，read_only = True，context = self.context）
 return serialized_data.data 
 
 class Meta：
 model = Parent 
  

加上Grandaughter，Grandson，你的细节代码，但我想你得到的照片。

当我运行我的父列表视图，并且我监视数据库查询时，我会得到一些沿着1000个查询的方式，只有少数的父母。 / p>

如果我在django shell中运行相同的代码，我可以在不超过25个查询中完成相同的查询。我怀疑也许这与我使用django多态库有关？原因是，有一个儿童和GrandChild数据库表，除了每个儿子/女儿，孙子/孙女表，总共六张桌子。跨越这些对象。所以我的直觉告诉我我错过了这些多态表。

或者，我的daata模型还有一个更优雅的解决方案？

解决方案

据我所知，嵌套的序列化程序可以访问预取关系，只需确保不修改查询集（即使用 all ））

  class ParentSerializer（serializer.ModelSerializer）：
 child = serializers。 SerializerMethodField（'get_children_ordered'）
 
 def get_children_ordered（self，parent）：
＃all（）调用应该缓存
 serialized_data = ChildSerializer（parent.child.all ），many = True，read_only = True，context = self.context）
 return serialized_data.data 
 
 class Meta：
 model = Parent 
 
 
 class ParentList（generics.ListAPIView）：
 
 def get_queryset（self）：
 children = Prefetch（'child'，queryset = Child.objects.select_相关（'parent'））
 queryset = Parent.objects.prefetch_related（children）
 return queryset 
 
 serializer_class = ParentSerializer 
 permission_classes =（permissions.IsAuthenticated，） 
  

I have a situation where my model has a Foreign Key relationship:

# models.py
class Child(models.Model):
    parent = models.ForeignKey(Parent,)

class Parent(models.Model):
    pass

and my serializer:

class ParentSerializer(serializer.ModelSerializer):
    child = serializers.SerializerMethodField('get_children_ordered')

    def get_children_ordered(self, parent):
        queryset = Child.objects.filter(parent=parent).select_related('parent')
        serialized_data = ChildSerializer(queryset, many=True, read_only=True, context=self.context)
        return serialized_data.data

    class Meta:
        model = Parent

When I call Parent in my views for N number of Parents, Django does N number of database calls inside the serializer when it grabs the children. Is there any way to get ALL children for ALL Parents to minimize the number of database calls?

I've tried this but it doesn't seem to solve my issue:

class ParentList(generics.ListAPIView):

    def get_queryset(self):
        queryset = Parent.objects.prefetch_related('child')
        return queryset

    serializer_class = ParentSerializer
    permission_classes = (permissions.IsAuthenticated,)

EDIT

I've updated the code below to reflect Alex's feedback....which solves the N+1 for one nested relationship.

# serializer.py
class ParentSerializer(serializer.ModelSerializer):
    child = serializers.SerializerMethodField('get_children_ordered')

    def get_children_ordered(self, parent):
        # The all() call should hit the cache
        serialized_data = ChildSerializer(parent.child.all(), many=True, read_only=True, context=self.context)
        return serialized_data.data

    class Meta:
            model = Parent

# views.py
class ParentList(generics.ListAPIView):

    def get_queryset(self):
        children = Prefetch('child', queryset=Child.objects.select_related('parent'))
        queryset = Parent.objects.prefetch_related(children)
        return queryset

    serializer_class = ParentSerializer
    permission_classes = (permissions.IsAuthenticated,)

Now let's say I have one more model, which is a grandchild:

# models.py
class GrandChild(models.Model):
    parent = models.ForeignKey(Child,)

class Child(models.Model):
    parent = models.ForeignKey(Parent,)

class Parent(models.Model):
    pass

If i place the following in my views.py for the Parent queryset:

queryset = Parent.objects.prefetch_related(children, 'children__grandchildren')

It doesn't look like those grandchildren are being carried on into the ChildSerializer, and thus, again I'm running another N+1 issue. Any thoughts on this one?

EDIT 2

Perhaps this will provide clarity...Maybe the reason i am still running into N + 1 database calls, is because both my children and grandchildren classes are Polymorphic.... i.e.

# models.py
class GrandChild(PolymorphicModel):
    child = models.ForeignKey(Child,)

class GrandSon(GrandChild):
    pass

class GrandDaughter(GrandChild):
    pass

class Child(PolymorphicModel):
    parent = models.ForeignKey(Parent,)

class Son(Child):
    pass

class Daughter(Child):
    pass

class Parent(models.Model):
    pass

and my serializers look more like this:

# serializer.py
class ChildSerializer(serializer.ModelSerializer):
    grandchild = serializers.SerializerMethodField('get_children_ordered')

    def to_representation(self, value):
        if isinstance(value, Son):
            return SonSerializer(value, context=self.context).to_representation(value)
        if isinstance(value, Daughter):
            return DaughterSerializer(value, context=self.context).to_representation(value)

    class Meta:
        model = Child

class ParentSerializer(serializer.ModelSerializer):
    child = serializers.SerializerMethodField('get_children_ordered')

    def get_children_ordered(self, parent):
        queryset = Child.objects.filter(parent=parent).select_related('parent')
        serialized_data = ChildSerializer(queryset, many=True, read_only=True, context=self.context)
        return serialized_data.data

    class Meta:
        model = Parent

Plus the same for Grandaughter, Grandson, I'll spare you the details codewise, but i think you get the picture.

When i run my view for ParentList, and i monitor DB queries, I'm getting something along the lines of 1000s of queries, for only a handful of parents.

If i run the same code in the django shell, i can accomplish the same query at no more than 25 queries. I suspect maybe it has something to do with the fact that I'm using the django-polymorphic library? The reason being is that, there's a Child and GrandChild database table, in additions to each Son/Daughter, Grandson/Granddaughter table, for a total of 6 tables. across those objects. So my gut tells me i'm missing those polymorphic tables.

Or perhaps there's a more elegant solution for my daata model?

解决方案

As far as I remember, nested serializers have access to prefetched relations, just make sure you don't modify a queryset (i.e. use all()):

class ParentSerializer(serializer.ModelSerializer):
    child = serializers.SerializerMethodField('get_children_ordered')

    def get_children_ordered(self, parent):
        # The all() call should hit the cache
        serialized_data = ChildSerializer(parent.child.all(), many=True, read_only=True, context=self.context)
        return serialized_data.data

    class Meta:
            model = Parent


class ParentList(generics.ListAPIView):

    def get_queryset(self):
        children = Prefetch('child', queryset=Child.objects.select_related('parent'))
        queryset = Parent.objects.prefetch_related(children)
        return queryset

    serializer_class = ParentSerializer
    permission_classes = (permissions.IsAuthenticated,)             

这篇关于Django REST Serializer做N + 1数据库调用多个嵌套关系，3个级别的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！