领域与Firestore上的大型馆藏同步-体系结构问题 [英] Realm syncing with large collection on Firestore - architectural questions / issues

问题描述

我在Firestore集合中有一个产品对象集合(标题，说明，价格，数量，urlString等).目前大约有1000件商品，但可能会达到1万件.在启动iOS应用程序时，我设置了一个集合侦听器(db.collection("products").rx.listen())，然后将更改保存到本地Realm数据库中.

I have a collection of product objects (title, desc, price, quant, urlString, etc) in a Firestore collection. Currently around 1000 items, but that could go to 10k. On my iOS app launch, I setup a collection listener (db.collection("products").rx.listen()) which then saves changes to a local Realm database.

.subscribe(onNext: { querySnapshot in
     querySnapshot.documentChanges.forEach { docChange in
                autoreleasepool {
                    let realm = try! Realm(configuration: Realm.Configuration.defaultConfiguration)
                    let newData = docChange.document.data()
                    if let item = itemFactory.createItem(using: newData) {
                        if (docChange.type == .added) {
                            //realm.add(item)
                        }
                        if (docChange.type == .modified) {
                            //realm.update(item)
                        }
                        if (docChange.type == .removed) {
                            //realm.delete(item)
                        }
                    }
                }
            }
        }, onError: { error in
            print("Error fetching snapshots: \(error)")
        }).disposed(by: disposeBag)

我已经详细阅读了firestore文档，但我不是100％确信这种方法可靠或高效.

I've read the firestore docs in detail but I'm not 100% confident this approach is reliable or performant.

问题:应用启动后，Firestore会在描述更改之前每次下载所有10k文档吗?还是将它第一次缓存全部10k，然后仅在后续启动时下载更改.我有信心一旦发生变更事件，所有后续事件将仅对Firestore数据库进行变更.我担心的是，在应用启动时首先订阅了侦听器，它会下载所有1万个项目(这会很昂贵).

Question: When the app launches, will Firestore download all 10k documents each time, before describing the changes? Or will it cache all 10k the very first time then only download changes on subsequent launches. I'm confident once a change event has fired, all subsequent events will only pick up changes to the Firestore database. What I'm concerned about is on first subscribing to the listener when the app launches, it downloads all 10k items (which would be costly).

编辑2019年1月9日:

我最终用大致如下的代码实现了@zavtra优雅的答案:

I ended up implementing @zavtra elegant answer with code roughly looking like this:

var newestUpdatedAt = UserDefaults.standard.double(forKey: kUDItemUpdatedAt)
//...
db.collection(kProducts)
            .whereField(kUpdatedAt, isGreaterThan: newestUpdatedAt)
            .rx.listen()
//...

querySnapshot.documentChanges.forEach { docChange in
            autoreleasepool {
                let realm = try! Realm(configuration: Realm.Configuration.defaultConfiguration)
                let newData = docChange.document.data()
                if let item = itemFactory.createItem(using: newData) {
                    if item.updatedAt > newestUpdatedAt {
                       newestUpdatedAt = item.updatedAt
                    }
                    if (docChange.type == .added) {
                        //realm.add(item)
                    }
                    if (docChange.type == .modified) {
                        //realm.update(item)
                    }
                    if (docChange.type == .removed) {
                        //realm.delete(item)
                    }
                }
            }
        }
        UserDefaults.standard.set(newestUpdatedAt, forKey: kUDItemUpdatedAt)

推荐答案

根据文档:

docChanges返回自上一次以来的文档更改的数组 快照.如果这是第一个快照，则所有文档都将位于 列为已添加"更改.

docChanges returns an array of the document changes since the last snapshot. If this is the first snapshot, all documents will be in the list as "added" changes.

每次重新启动应用程序时，都会触发此第一张快照"行为.如果要解决此问题，则必须:

Every time you re-start the app will trigger this "first snapshot" behavior. If you want to get around this behavior, you would have to:

1. 检索本地保存的最新文档及其时间戳.
2. 构建一个查询，其中所有文档均从该时间戳记开始(即每个文档的时间戳记至少为最新保存的时间戳记)
3. 在应用程序条目上订阅有关该查询的更改.

要执行此操作，您将必须向每个文档添加一个时间戳字段，并且在Firestore规则中的时间戳字段上添加一个"indexOn"规则，以防止客户端对整个集合进行下载和排序.

To do this, you will have to add a timestamp field to each document, and an "indexOn" rule in your firestore rules on the timestamp field in order to prevent client-side downloading and sorting of the entire collection.

这篇关于领域与Firestore上的大型馆藏同步-体系结构问题的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！