在Google App Engine的for循环中查询的有效方法？ [英] Efficient way to query in a for loop in Google App Engine?

问题描述

在GAE文档中，它指出：

lockquote
因为每个get（）或put（）操作调用一个单独的远程
过程调用（RPC），在循环内发出很多这样的调用是一种
低效的方式来同时处理一组实体或键。

谁知道我的代码中有多少其他的低效率问题，所以我想尽量减少。目前，我确实有一个for循环，每个迭代都有一个单独的查询。假设我有一个用户，并且一个用户有朋友。我想获取用户的每个朋友的最新更新。因此，我拥有的是该用户朋友的数组：

 对于朋友中的friend_dic：
 email = friend_dic [' email'] 
 lastUpdated = friend_dic ['lastUpdated'] 
 userKey = Key（'User'，email）
 query = ndb.gql（'SELECT * FROM StatusUpdates WHERE ANCESTOR IS：1 AND （））：
 status = qit.next（）
 status_list.append（status.to_dict（））
 raise ndb.Return（status_list）
  

有没有一种更有效的方式来做到这一点，也许不知何故将所有这些批处理成一个单一的查询？

解决方案

尝试查看NDB地图功能： https://developers.google.com/appengine/docs/蟒蛇/ NDB / queryclass＃Query_map_async

示例（假设您将您的朋友关系保留在单独的模型中，对于此示例，我假定关系
pre $
$ b $ def $ callback（entity）
email = friend_dic ['email']
lastUpdated = friend_dic ['lastUpdated']
userKey = Key（'User'，email）
query = ndb.gql（'SELECT * FROM StatusUpdates WHERE ANCESTOR IS： 1 AND modifiedDate> ：2'，userKey，lastUpdated）
status_updates = yield query.fetch_async（）
raise ndb.Return（status_updates）
$ b qry = ndb.gql（SELECT * FROM Relationships WHERE friend_to = 1，user.key）
updates = yield qry.map_async（回调）
#updates现在是一个状态更新列表


更新：

更好地理解您的数据模型：

 查询= [] 
 status_list = [] 
在朋友中为friend_dic：
 email = friend_dic ['email'] 
 lastUpdated = friend_dic ['lastUpdated'] 
 userKey = Key（'User'，email）
 queries.append（ndb.gql（'SELECT * FROM StatusUpdates WHERE ANCESTOR IS：1 AND modifiedDate >：2'，userKey，lastUpdated）.fetch_async（））
 
用于查询查询：
 status = yield query 
 status_list.extend（[x.to_dict（） for x in status]）
 
 raise ndb.Return（status_list）
  

In the GAE documentation, it states:

Because each get() or put() operation invokes a separate remote procedure call (RPC), issuing many such calls inside a loop is an inefficient way to process a collection of entities or keys at once.

Who knows how many other inefficiencies I have in my code, so I'd like to minimize as much as I can. Currently, I do have a for loop where each iteration has a separate query. Let's say I have a User, and a user has friends. I want to get the latest updates for every friend of the user. So what I have is an array of that user's friends:

for friend_dic in friends:
        email = friend_dic['email']
        lastUpdated = friend_dic['lastUpdated']
        userKey = Key('User', email)
        query = ndb.gql('SELECT * FROM StatusUpdates WHERE ANCESTOR IS :1 AND modifiedDate > :2', userKey, lastUpdated)
        qit = query.iter()
        while (yield qit.has_next_async()):
           status = qit.next()
           status_list.append(status.to_dict())
raise ndb.Return(status_list)

Is there a more efficient way to do this, maybe somehow batch all these into one single query?

解决方案

Try looking at NDB's map function: https://developers.google.com/appengine/docs/python/ndb/queryclass#Query_map_async

Example (assuming you keep your friend relationships in a separate model, for this example I assumed a Relationships model):

@ndb.tasklet
def callback(entity):
  email = friend_dic['email']
  lastUpdated = friend_dic['lastUpdated']
  userKey = Key('User', email)
  query = ndb.gql('SELECT * FROM StatusUpdates WHERE ANCESTOR IS :1 AND modifiedDate > :2', userKey, lastUpdated)
  status_updates = yield query.fetch_async()
  raise ndb.Return(status_updates)

qry = ndb.gql("SELECT * FROM Relationships WHERE friend_to = :1", user.key)
updates = yield qry.map_async(callback)
#updates will now be a list of status updates

Update:

With a better understanding of your data model:

queries = []
status_list = []
for friend_dic in friends:
  email = friend_dic['email']
  lastUpdated = friend_dic['lastUpdated']
  userKey = Key('User', email)
  queries.append(ndb.gql('SELECT * FROM StatusUpdates WHERE ANCESTOR IS :1 AND modifiedDate > :2', userKey, lastUpdated).fetch_async())

for query in queries:
  statuses = yield query
  status_list.extend([x.to_dict() for x in statuses])

raise ndb.Return(status_list)

这篇关于在Google App Engine的for循环中查询的有效方法？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！