在Google App Engine中建立分片列表 [英] Building a sharded list in Google App Engine
问题描述
我正在寻找一种好的设计模式来在Google App Engine中分片列表.我已按照Google文档此处所述，了解并实现了分片计数器.但我现在正尝试将相同的原理应用于列表.以下是我的问题和可能的解决方案-请输入您的意见?
I am looking for a good design pattern for sharding a list in Google App Engine. I have read about and implemented sharded counters as described in the Google Docs here but I am now trying to apply the same principle to a list. Below is my problem and possible solution - please can I get your input?
问题: 我系统上的用户可能会收到许多类似于在线聊天系统的消息.我希望服务器记录所有传入的消息(它们将包含多个字段-从，到等).但是，从文档中我知道，更新同一实体组通常会导致由数据存储争用引起的异常.当一个用户在短时间内收到许多消息，从而导致其实体被多次写入时，可能会发生这种情况.那么，如何抽象出上面的分片计数器示例呢?
Problem: A user on my system could receive many messages kind of like a online chat system. I'd like the server to record all incoming messages (they will contain several fields - from, to, etc). However, I know from the docs that updating the same entity group often can result in an exception caused by datastore contention. This could happen when one user receives many messages in a short time thus causing his entity to be written to many times. So what about abstracting out the sharded counter example above:
- 定义五个实体/实体组
- 对于要添加的每条消息，随机选择一个实体，然后将消息附加到该实体上，再将其写回到商店中，
- 要获取消息列表，请阅读中的所有实体并合并...
对上述问题进行一些提问:
Ok some questions on the above:
- 最重要的是，这是处理事情的最佳方法吗?还是存在更优雅/更有效的设计模式?
- 哪种方法可以有效过滤其中某个字段说出某个特定日期之后的所有内容的邮件列表?
- 如果我需要一个分片集怎么办?我应该阅读所有实体并检查每次写入是否都已存在新项目吗?还是只是按上述方式添加它，然后在下一个请求读取时删除重复项?
推荐答案
为什么要将所有消息放在1个实体组中?
why would you want to put all messages in 1 entity group ?
如果不指定祖先，则不需要分片，但是由于最终的一致性，最终用户在查询消息时可能会看到一些滞后.
If you don't specify a ancestor, you won't need sharding, but the end user might see some lagging when querying the messages due to eventual consistency.
取决于是否可以接受权衡.
Depends if that is an acceptable tradeoff.
这篇关于在Google App Engine中建立分片列表的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
