用于同步大量数据的 DDP 与直接 MongoDB 访问 [英] DDP vs Straight MongoDB access for synching large amounts of data

问题描述

我们正在 Meteor 中构建一个应用程序，该应用程序将参与教育生态系统.有许多应用程序(例如成绩簿、学生信息系统、报告系统...)都需要将它们的数据存储与 Meteor 保持同步.数据存储大小将在数十万个文档中.

We are building an app in Meteor that will be participating in an education ecosystem. There are a number of applications (e.g. a GradeBook, a Student Information System, a Reporting System...) that will all need to have their data stores kept in synch with Meteor. The datastore size will be in the hundreds of thousands of documents.

我的理解是，DDP 用于将客户端"连接到 Meteor 应用程序(通过在 Meteor 推送数据更改和 RPC 以将数据导入 Meteor 时订阅提要).并且客户端"通常仅限于用户……因此与整个数据集相比，数据集的大小相对较小(教师可能可以访问 250K 文档中的 100 个).

My understanding is that DDP is used to connect "clients" to a Meteor app (by subscribing to feeds when Meteor is pushing data changes and RPC to get the data in to Meteor). And a "client" is generally scoped to a user...so the size of the data set is relatively small compared to the universe of data (a teacher might have access to 100 of the 250K documents).

如果我使用 DDP 将报告系统(作为客户端")连接到 Meteor，存储中的所有数据都需要同步...这是否意味着每次报告系统失去与 Meteor 的连接时，所有数据会从 Meteor 重新发送到 DDP 客户端吗?(因为报告系统对所有数据都感兴趣)......如果是这样，DDP 就不是保持应用程序同步的方式，对吗?......它对于范围更小的数据集意味着更多........而且我们可能应该直接与 Mongo 交互以保持同步.

If I connected a Reporting System (as a "client") to Meteor with DDP, all data in the store would need to be synched...does that mean that every time the Reporting System lost the connection to Meteor, all data would be re-sent from Meteor to the DDP client? (because the Reporting System is interested in ALL the data)...and if that's the case, DDP wouldn't be the way to keep application in synch, right?...it's meant more for much smaller scoped data sets....and we should probably be interacting directly with Mongo to keep things synch.

谢谢！迈克

推荐答案

我认为 nate 的回答非常适合您应该做什么，特别是考虑到数据量.如果您需要显示大量数据，如果您使用页面来使用分页订阅，这样您就可以享受实时功能(如果您决定使用它)，而无需一次性下载所有内容.请记住，目前数据是这样发送的(对于每个会话，因此如果选项卡关闭并重新打开，它将重做):

I think nate's answer goes perfect on what you should do especially considering the volume of data. And if you need to display a whole lot of data if you're using pages to use a paginated subscription so that you can enjoy the realtime functionality (if you decide to use it) without downloading it all at once. Keep in mind though that at the moment the data is sent down like this (for each session, so if the tab is closed and reopened it will be redone):

1 - Connect to DDP Server/Proxy (Long Polling now due to websocket issues with chrome)
2 - Establish a 'subscription'
3 - Fetch all data relevant to subscription (initial download)
4 - Subscription is complete, now the client will 'listen' for changes
5 - Any updates (remove/update/insert, etc) are sent down to the client

此时确实没有同步系统可以将旧数据保持离线状态(在本地存储或索引数据库或任何地方)，因此可以避免第 3 步，并且只会发生从该点开始的同步.

There really isn't a sync system at this point where old data is kept offline (in a localstorage or indexed db or anything) so that step no 3 can be avoided and only the syncs from that point would occur.

请注意，如果连接中断(例如，在短时间内失去连接性，Meteor 将失去与 DDP 线的连接，当它重新连接时，它会像从头开始一样重新下载所有内容.

This is mind, if there is a connectivity interruption (e.g losing connectivitiy for a short peroid of time Meteor will lose connection to the DDP wire and when it reconnects it download everything again as if it were from scratch.

这篇关于用于同步大量数据的 DDP 与直接 MongoDB 访问的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！