超过Firestore截止期限节点 [英] Firestore Deadline Exceeded Node

问题描述

我想加载约3万条记录的集合.即通过加载它.

I would like to load collection that is ~30k records. I.e load it via.

const db = admin.firestore();
let documentsArray: Array<{}> = [];
db.collection(collection)
  .get()
  .then(snap => {
    snap.forEach(doc => {
      documentsArray.push(doc);
    });
  })
  .catch(err => console.log(err));

这将始终引发Deadline Exceeded错误.我已经搜索了某种机制，可以让我对其进行分页，但我发现无法一次性查询这么多的金额真是令人难以置信.

This will always throw Deadline Exceeded error. I have searched for some sorts of mechanism that will allow me to paginate trough it but I find it unbelievable not to be able to query for not that big amount in one go.

我当时在想，可能是由于我的机器速度较慢，我已经达到了极限，但是后来我部署了一个简单的Express应用程序，该应用程序可以抓取到应用程序引擎，但仍然没有运气.

I was thinking that it may be that due to my rather slow machine I was hitting the limit but then I deployed simple express app that would do the fetching to app engine and still had no luck.

或者，我也可以使用gcloud beta firestore export导出集合，但是它不提供JSON数据.

Alternatively I could also export the collection with gcloud beta firestore export but it does not provide JSON data.

推荐答案

最后，问题是正在处理来自Firestore的30k记录的计算机的功能不足，无法及时获取所需的数据.通过使用GCE和n1-standard-4 GCE来解决.

In the end the issue was that machine that was processing the 30k records from the Firestore was not powerful enough to get the data needed in time. Solved by using, GCE with n1-standard-4 GCE.

这篇关于超过Firestore截止期限节点的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！