DDP与直接MongoDB访问可同步大量数据 [英] DDP vs Straight MongoDB access for synching large amounts of data

问题描述

我们正在Meteor中构建一个应用程序，该应用程序将参与教育生态系统. 有许多应用程序(例如，成绩簿，学生信息系统，报告系统...)都需要使其数据存储与Meteor保持同步.数据存储区的大小将达到数十万个文档.

We are building an app in Meteor that will be participating in an education ecosystem. There are a number of applications (e.g. a GradeBook, a Student Information System, a Reporting System...) that will all need to have their data stores kept in synch with Meteor. The datastore size will be in the hundreds of thousands of documents.

我的理解是DDP用于将客户端"连接到Meteor应用程序(在Meteor推送数据更改和RPC以便将数据输入Meteor时订阅订阅源).通常，客户端"的作用范围是用户...因此，与数据的整体范围相比，数据集的大小相对较小(老师可以访问250K文档中的100个).

My understanding is that DDP is used to connect "clients" to a Meteor app (by subscribing to feeds when Meteor is pushing data changes and RPC to get the data in to Meteor). And a "client" is generally scoped to a user...so the size of the data set is relatively small compared to the universe of data (a teacher might have access to 100 of the 250K documents).

如果我使用DDP将报告系统(作为客户端")连接到Meteor，则存储中的所有数据都需要同步...这是否意味着每次Reporting System失去与Meteor的连接时，所有数据会从Meteor重新发送到DDP客户端? (因为报表系统对所有数据都感兴趣)...如果是这种情况，DDP不会成为保持应用程序同步的方法，对吗?...对于更小范围的数据集而言，它意味着更多. ..我们应该直接与Mongo互动以保持事物同步.

If I connected a Reporting System (as a "client") to Meteor with DDP, all data in the store would need to be synched...does that mean that every time the Reporting System lost the connection to Meteor, all data would be re-sent from Meteor to the DDP client? (because the Reporting System is interested in ALL the data)...and if that's the case, DDP wouldn't be the way to keep application in synch, right?...it's meant more for much smaller scoped data sets....and we should probably be interacting directly with Mongo to keep things synch.

谢谢！ 迈克

推荐答案

我认为nate的答案非常适合您应该做的事情，尤其是考虑到数据量.而且，如果您要使用页面来使用分页订阅，则需要显示大量数据，这样您就可以享受实时功能(如果您决定使用它)而无需一次下载所有内容.请记住，尽管此刻数据是这样发送的(对于每个会话，所以如果关闭并重新打开选项卡，它将被重做):

I think nate's answer goes perfect on what you should do especially considering the volume of data. And if you need to display a whole lot of data if you're using pages to use a paginated subscription so that you can enjoy the realtime functionality (if you decide to use it) without downloading it all at once. Keep in mind though that at the moment the data is sent down like this (for each session, so if the tab is closed and reopened it will be redone):

1 - Connect to DDP Server/Proxy (Long Polling now due to websocket issues with chrome)
2 - Establish a 'subscription'
3 - Fetch all data relevant to subscription (initial download)
4 - Subscription is complete, now the client will 'listen' for changes
5 - Any updates (remove/update/insert, etc) are sent down to the client

在这一点上，确实没有同步系统，旧数据保持脱机状态(在本地存储或索引数据库或其他任何数据中)，因此可以避免第3步，并且只有从该点开始的同步.

There really isn't a sync system at this point where old data is kept offline (in a localstorage or indexed db or anything) so that step no 3 can be avoided and only the syncs from that point would occur.

请记住，如果出现连接中断(例如，在短时间内失去连接性，Meteor将失去与DDP线的连接，并且当它重新连接时，它将重新下载所有内容，就像从头开始一样.

This is mind, if there is a connectivity interruption (e.g losing connectivitiy for a short peroid of time Meteor will lose connection to the DDP wire and when it reconnects it download everything again as if it were from scratch.

这篇关于DDP与直接MongoDB访问可同步大量数据的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！