当只有约 50 个实体时，GAE Python NDB 查询获取响应时间 >55 秒 [英] GAE Python NDB query fetch response time >55 seconds when only ~50 entities are present

问题描述

我们有一个旅游搜索网站.为了搜索公交车，我们对公交车运营商实体执行查询.我们使用 GAE Python NDB.当仅存在约 50 个实体时，查询获取响应时间 >55 秒(在部署版本而非开发服务器上).

We have a travel search website. To search for a bus, we execute a query on the bus operator entities. We use GAE Python NDB. The query fetch response time >55 seconds(on deployed version and not development server) when only ~50 entities are present.

目前我的查询包含五个项目.如果我增加到五个以上，响应会进一步减慢.请提出将查询时间缩短到约 1 或 2 秒或尽可能短的方法.
请在下面找到相关详细信息(对不起，我试图在某种程度上最小化以下内容):

Presently my query contains five items. If I increase to more than five, the response slows further considerably. Please suggest ways to bring down the query time to ~1 or 2 seconds or as less as possible.
Please find the relevant details below(sorry i tried to minimize the below content to some extent):

查询代码:

start_time = datetime.datetime.now() 
qry_1 = X.query(ndb.AND(X.active_status=="active", X.property_3==input_3, X.property_4==input_4, X.property_5==input_5, X.property_6.IN(input_6_list), X.property_20.IN(input_20_list))) 
record_list = qry_1.fetch() 
query_end_time = datetime.datetime.now() 
query_execution_time = query_end_time - start_time 
logging.info ("query_execution_time=["+str(query_execution_time)+"] ") 

# input_6_list contains ~5 string items 
# input_20_list contains ~5 string items 

日志输出:

query_execution_time=[0:00:55.925250]  

实体模型:

class X(ndb.Model): 
    active_status = ndb.StringProperty() 
    name = ndb.StringProperty() 
    property_1 = ndb.StringProperty() 
    property_2 = ndb.TextProperty() 
    property_3 = ndb.StringProperty() 
    property_4 = ndb.StringProperty() 
    property_5 = ndb.StringProperty() 
    property_6 = ndb.StringProperty() 
    property_7 = ndb.StringProperty() 
    property_8 = ndb.StringProperty() 
    property_9 = ndb.StringProperty(repeated=True) 
    property_10 = ndb.StringProperty(repeated=True) 
    property_11 = ndb.StringProperty() 
    property_12 = ndb.StructuredProperty(P) 
    property_13 = ndb.StructuredProperty(Q) 
    property_14 = ndb.StringProperty() 
    property_15 = ndb.StructuredProperty(R, repeated=True) 
    property_16 = ndb.StructuredProperty(S, repeated=True) 
    property_17 = ndb.StringProperty() 
    property_18 = ndb.StringProperty(repeated=True) 
    property_19 = ndb.StringProperty() 
    property_20 = ndb.StringProperty(repeated=True) 
    property_21 = ndb.StringProperty(repeated=True) 
    property_22 = ndb.StructuredProperty(T, repeated=True) 
    property_23 = ndb.IntegerProperty(default=6) 
    property_24 = ndb.IntegerProperty(default=6) 
    property_25 = ndb.IntegerProperty(default=6) 
    property_26 = ndb.IntegerProperty(default=6) 
    property_27 = ndb.IntegerProperty(default=6) 
    property_28 = ndb.IntegerProperty(default=0) 
    property_29 = ndb.IntegerProperty() 
    date_added = ndb.DateTimeProperty(auto_now_add=True) #creation date 
    date_modified = ndb.DateTimeProperty(auto_now=True) #update date 
    property_30 = ndb.TextProperty() 
    property_31 = ndb.BlobKeyProperty() 
    property_32 = ndb.BlobKeyProperty() 
    property_33 = ndb.BlobKeyProperty() 
    property_34 = ndb.BlobKeyProperty() 
    property_35 = ndb.BlobKeyProperty() 
    property_36 = ndb.BlobKeyProperty() 
    property_37 = ndb.BlobKeyProperty() 
    property_38 = ndb.StringProperty() 
    property_39 = ndb.BlobKeyProperty() 
    property_40 = ndb.StringProperty(default="not_allowed")  

在调试此问题时，我运行了 Appstats 并遇到了另一个问题 这是我在 SO 上问的

While debugging this issue, I ran Appstats and had another question which I asked on SO

推荐答案

过滤额外的属性通常并不昂贵.但是使用IN"是.2 个包含 5 个项目列表的 IN 过滤器需要在后端进行 25 次并发搜索.

Filtering on additional properties is not usually expensive. But using 'IN' is. 2 IN filters with lists of 5 items each requires 25x concurrent seeks on the backend.

你能从你的代码目录中发布 index.yaml 文件吗?如果此文件不存在，则查询将需要多个 JOIN，这可以解释速度缓慢的原因.对 dev_appserver 运行相同的查询，它将自动生成文件.

Could you post the index.yaml file from your code directory? If this file does not exist, the query would require multiple JOINs, which would explain the slowness. Run the same query against the dev_appserver and it will generate the file automatically.

更多信息:https://developers.google.com/appengine/docs/python/config/indexconfig

顺便说一下，在您不打算搜索的属性上使用indexed=False"将大大降低投入成本.

By the way, using 'indexed=False' on the properties you don't intend to search on will reduce the cost puts considerably.

这篇关于当只有约 50 个实体时，GAE Python NDB 查询获取响应时间 >55 秒的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！