从Django的RawQuerySet获取数据库游标 [英] Get database cursor from Django's RawQuerySet

问题描述

我在Postgres中有一个相当大的表（20M +），我尝试用一​​个原始的Django查询：

  tweets = TweetX.objects.raw（SELECT * from twitter_tweet）。using（twittertest）
  

我快速获得了一个RawQuerySet，但是当我尝试迭代其结果时，它会停止播放：

 用于tweet的tweet ：
 #do stuff 
  

内存正在稳步上升，所以我怀疑整个数据集都被转移。
有没有办法从 .raw 获取数据库游标，所以我可以迭代结果集，而不是一次传输它？

解决方案

似乎很难说服django / postgres使用数据库游标。相反，它会获取所有内容，然后在其上放置一个客户端迭代器（称为光标）。

找到解决方案 over 这里显式创建数据库游标。只有不足之处在于它不适合django模型。

 从django.db导入连接
 
 conn = connections ['twittertest'] 
＃如果conn.connection是None，则需要正确填充连接对象
 
 cursor = conn.cursor（）
 
 cursor = conn.connection.cursor（name ='gigantic_cursor'）
 cursor.execute（SELECT * from twitter_tweet）
 
用于光标中的推文：
 #profit 
  

I have a sizable table (20M+) in Postgres am I try to do a raw Django query on it:

tweets = TweetX.objects.raw("SELECT * from twitter_tweet").using("twittertest")

I get a RawQuerySet fast, but when I try to iterate over its results it is grinding to a halt:

for tweet in tweets:
   #do stuff

Memory is steadily rising so I suspect the whole dataset is being transferred. Is there a way to get a database cursor from .raw so I can iterate over the result set without transferring it all at once?

解决方案

It seems that it is rather difficult to persuade django/postgres to use database cursors. Instead it fetches everything and then put a client side iterator (called cursor) over it.

Found a solution over here that explicitly creates a db cursor. Only downside is it does not fit into django models anymore.

from django.db import connections

conn = connections['twittertest']
# This is required to populate the connection object properly
if conn.connection is None:
    cursor = conn.cursor()        

cursor = conn.connection.cursor(name='gigantic_cursor')
cursor.execute("SELECT * from twitter_tweet")

for tweet in cursor:
    #profit

这篇关于从Django的RawQuerySet获取数据库游标的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！