如何在谷歌应用程序引擎中执行大型查询？ [英] How do I perform a large query in google app engine?
问题描述
我有大约100万用户的应用程序集合。我现在使用Google云消息添加推送通知系统来创建警报。我的数据库包含一个具有GcmId和应用程序名称的实体（例如myApp1）。
现在，我想向所有myApp1用户发送GCM消息。尽管如此，客体化文档并没有很好地描述.limit函数。例如，从GCM演示程序：
List< RegistrationRecord>记录= ofy（）.load（）.type（RegistrationRecord.class）.limit（10）.list（）;
会发送到前10个条目。但我需要匹配appType =myApp1的所有条目。这很难，因为查询可能很大，并可能匹配50万用户，我需要将GCM推送到所有人。
这么大查询执行？
编辑：我目前正在使用
List< RegistrationRecord>记录= ofy（）.load（）.type（RegistrationRecord.class）.filter（app，myApp1）.list（）;
进行测试，并且工作正常。然而，当推送活时，数据集是巨大的，我不知道什么是影响。
我相信你是从错误的角度来看。
对象化或低级别的appengine可以很好地处理使用游标的分页结果，因此您需要通过块处理结果。我不会详细介绍如何做到这一点，因为这将最终花费你所有这些数据存储读取大量的$，你将需要任务队列。
相反，查看Google云消息传递中的主题：
https://developers.google.com /云消息传递/主题消息传递
用户（客户端应用程序）订阅主题（您的案例中的appid）。然后你发送一个单一的主题推送，从一个appengine前端实例（限于30秒响应等）中更容易。
我发现这篇博文很棒一个完整实现的例子，以及如何正确处理可能的错误：
我能看到的唯一问题是从服务器的推送记录最多需要30秒。一个appengine前端实例也有30秒的限制总数，所以当它等待GCM推送完成时，servlet本身可能会超时。解决这个问题的一种方法是发送来自taskqueue的推送，这会给你60秒的urlfetch调用（我认为该限制也适用于任何api调用）： https://cloud.google.com/appengine/docs/java/urlfetch/
I have collection of apps that total almost 1 million users. I am now adding a push notification system using Google cloud messaging to create alerts. My database contains an entity with the GcmId and app name (ex. "myApp1").
Now, I want to send a GCM message to all users of "myApp1". The objectify documents do not describe the .limit function well though. For example, from the GCM demo app:
List<RegistrationRecord> records = ofy().load().type(RegistrationRecord.class).limit(10).list();
will send to the first 10 entries. But I need all entries that match appType="myApp1". This is harder because the query can be large and could potentially match half a million users and I need to send the GCM push to all of them.
How is such a large query performed?
EDIT: I am currently using
List<RegistrationRecord> records = ofy().load().type(RegistrationRecord.class).filter("app","myApp1").list();
for testing and it is working fine. However when pushed live, the dataset is huge, and I don't know what the repercussions are.
i believe you are looking it from the wrong angle.
objectify or low level appengine deal very well with paginated results using cursors, so you need to process results by chunks. i wont go into details on how to do that because that would end up costing you a lot of $ for all those datastore reads and you will need task queues.
instead, look at topics in google cloud messaging: https://developers.google.com/cloud-messaging/topic-messaging
the user (client side app) subscribes to the topic (the appid in your case). you then send a single topic push which is much easier from an appengine frontend instance (limited to 30 seconds response and such).
I found this blog post to be a great example of a full implementation and how to properly handle possible errors:
the only issue i can see is that a push from server is documented to take up to 30 seconds. An appengine frontend instance also has a 30 seconds limit total, so while it waits for GCM push to complete the servlet itself can timeout. one way to solve this is to send the push from taskqueue which would give you 60 seconds for urlfetch calls (i assume that limit applies to any api call as well): https://cloud.google.com/appengine/docs/java/urlfetch/
这篇关于如何在谷歌应用程序引擎中执行大型查询？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
